Chapter5 Image Restoration - USTChome.ustc.edu.cn/~xuedixiu/image.ustc/course/dip/DIP14-ch5.pdf ·...

Chapter5 Image Restoration

• Preview5 1 I d i• 5.1 Introduction

• 5.2 Diagonalizationi d i ( i fil i )• 5.3 Unconstrained Restoration( inverse filtering)

• 5.4 Constrained Restoration( wiener filtering)• 5.5 Estimating the Degradation Function• 5.6 Geometric Distortion Correction• 5.7 image inpaiting

2003 Digital Image Processing Prof.Zhengkai Liu Dr.Rong Zhang

5.1 Introduction5.1.1 Purpose

"compensate for" or "undo" defects which degrade an image.

5.1.2 Degrade Causes

( ) h i b l

co pe s e o o u do de ec s w c deg de ge.

(1) atmospheric turbulence(2) sampling, quantization p254(3) motion blur(4) camera misfocus(5) noise

5.1 Introduction5.1.3 degradation model

n(x,y)

f( ) H g(x y)f(x,y) H + g(x,y)

Assume it is a linear, position- invariant system, We can model a blurred image by

( , ) ( , )* ( , ) ( , )g x y f x y h x y n x y= +

h h( ) i ll d i d i ( )Where h(x,y) is called as Point Spread Function (PSF)

∑∑− −

+−−=1 1

),(),(),(),(M N

yxnnymxhnmfyxg2003 Digital Image Processing

Prof.Zhengkai Liu Dr.Rong Zhang3

∑∑= =

),(),(),(),(m n

yxnnymxhnmfyxg

5.1 Introduction5.1.4 Methods

Unconstrained Restoration: inverse filteringConstrained Restoration: wiener filtering

5.1.5 problem expression

Estimate a true image f(x,y) from a degraded image g(x,y)b d i k l d f h( ) d h i i lbased on prior knowledge of PSF h(x,y)and the statisticalproperties of noise n(x,y)

5.2 Diagonalization 5.2.1 Matrix expression of degradation model: 1-D

( ) ( )* ( )g x f x h x=

( ) 0 1( )

f x x Af x

≤ ≤ −⎧= ⎨( )

0 elseef x = ⎨⎩

( ) 0 1( )

0 elsee

h x x Bh x

≤ ≤ −⎧= ⎨⎩ 0 else⎩

0( ) ( ) ( ) ( )

e e e eg x f m h x m n x−

= − +∑ x=0 1 M-1

M=A+B-1

⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤

0m= x 0,1,…,M 1

(0) (0) ( 1) ( 1) (0) (0)(1) (1) (0) ( 2) (1) (1)

e e e e e e

g h h h M f ng h h h M f n

g Hf n

− − +⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥− +⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥= + = = +⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥( -1) ( 1) ( 2) (0) ( -1) ( 1)e e e e e eg M h M h M h f M n M⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥− − −⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦

( ) ( )h x h x M= +( ) ( )e eh x h x M+(0) ( 1) (1)e e eh h M h−⎡ ⎤

⎢ ⎥(1) (0) (2)

( 1) ( 2) (0)

e e eh h hH

h M h M h

⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦( 1) ( 2) (0)e e eh M h M h⎢ ⎥− −⎢ ⎥⎣ ⎦

H is circulant

( , ) 0 1 and 0 1( )

f x y x A y Bf x

≤ ≤ − ≤ ≤ −⎧= ⎨( )

0 A 1 or B 1ef xx M y N

= ⎨ ≤ ≤ − ≤ ≤ −⎩

( , ) 0 1 and 0 1( )

h x y x C y Dh x

≤ ≤ − ≤ ≤ −⎧= ⎨( )

0 A 1 or B 1eh xx M y N

= ⎨ ≤ ≤ − ≤ ≤ −⎩

y=0 1 N-1

x=0,1,…,M-11 1

0 0( , ) ( , ) ( , )

e e em n

g x y f m n h x m y n− −

= − −∑∑y 0,1,…,N 1

(0) (0)(1) (1)

H H H f nH H H f n

g Hf n

−⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥= + = +⎢ ⎥ ⎢ ⎥ ⎢ ⎥

1 2 0 ( -1) ( 1)M M e e

H H H f MN n MN− −

⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥−⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦

H is block-circulant

( 0) ( 1) ( 1)h i h i N h i−⎡ ⎤

( ,0) ( , 1) ( ,1)( ,1) ( ,0) ( , 2)

e e ei

h i h i N h ih i h i h i

−⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥( , 1) ( , 2) ( ,0)e e eh i N h i N h i⎢ ⎥− −⎢ ⎥⎣ ⎦

5.2 Diagonalization 5.2.2 Diagonliztion: 1-D

The eigenvector and eigenvalue of a circulant matrix H are

2 2( ) 1 exp exp ( 1)T

w k j k j M kM Mπ π⎡ ⎤⎛ ⎞ ⎛ ⎞= −⎜ ⎟ ⎜ ⎟⎢ ⎥⎝ ⎠ ⎝ ⎠⎣ ⎦

M M⎝ ⎠ ⎝ ⎠⎣ ⎦

2 2( ) (0) ( 1)exp (1)exp ( 1)j jk h h M k h M kπ πλ ⎛ ⎞ ⎛ ⎞= + + +⎜ ⎟ ⎜ ⎟( ) (0) ( 1)exp (1)exp ( 1)e e ek h h M k h M kM M

λ = + − − + + − −⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠

Combine the M eigenvectors to a matrix

[ (0) (1) ( 1)]W w w w M= −g

then the H can be expressed as1H WDW −= ( , ) ( )D k k kλ=

then the H can be expressed aswhere

g Hf n= +for

W g W Hf W nW WDW f W n

− − −

= +1 1DW f W n− −= +

( ) ( ) ( ) ( )G u H u F u N u+( ) ( ) ( ) ( )G u H u F u N u= +

2( , ) exp NW i m j im WMπ⎛ ⎞= ⎜ ⎟

⎝ ⎠

2( , ) expNW k n j knNπ⎛ ⎞= ⎜ ⎟

⎝ ⎠

1H WDW −= ( , ) ( , ) ( , ) ( , )G u v H u v F u v N u v= +

g(x,y) = f(x,y)*h(x,y) +n(x,y) in Spatial Coordinates

G(u,v) = F(u,v)H(u,v) +N(u,v) in frequency domain

g = Hf + n in vector form

5.3 Inverse Filtering5.3 Inverse Filtering5.3.1 assumption

H i i d th i i li iblH is given, and the noise is negligible

5.3.2 degradation model

F(u,v) H G(u,v)

( , ) ( , ) ( , )G u v H u v F u v=

5.3 Inverse Filtering5.3 Inverse Filtering5.3.3 restoration

( , )ˆ ( , ) ( , ) ( , )( , ) I

G u vF u v G u v H u vH u v

1( , )( , )IH u v

H u v= deconvolution

5.3.4 properties ( , )ˆ ( , )( )

G u vF u vH u v

=( , )

( , ) ( , ) ( , )( , )

H u vH u v F u v N u v

H u v+

=( , )( , )( , )( , )

N u vF u vH u v

5.3 Inverse Filtering5.3 Inverse Filtering5.3.4 properties

sensitive to additive noise: if H(u,v) has zero or very small value,the N(u,v)/H(u,v) could easily dominate the estimate

5.3.5 improvements p2 2 2

01/ ( , ) if ( , )

1 elseH u v u v w

M u v⎧ + <

= ⎨⎩

H(u,v)⎩

or if ( , )

( , )1/ ( , ) elsek H u v d

M u vH u v

<⎧= ⎨⎩

5.3 Inverse Filtering5.3.4 examples: without noise

Original image Blurred image Restored image

5.3 Inverse Filtering5.3.4 examples: with noise

Blurred and noised image Restored image

5.3 Inverse Filtering5.3.4 examples: deferent cutoff

5 4 Wiener Filtering5.4 Wiener Filtering5.4.1 assumption

H is given, and consider the image and noise as random processes

5 4 2 request5.4.2 requestThe mean square error between uncorrupted image and estimated image is minimized This error measure is given byimage is minimized. This error measure is given by

{ }{ }2 2ˆ( )e E f f= −

5 4 Wiener Filtering5.4 Wiener Filtering5.4.3 restoration

ˆ ( , ) ( , ) ( , )WF u v G u v H u v=

Wi filt2

( , )1( , )( , ) ( , ) ( , ) / ( , )

H u vH u v

H u v H u v S u v S u v= ×

Wiener filter, 1942

2( , )

( , ) ( , ) / ( , )n f

H u vH u v S u v S u v

where2( , ) ( , )nS u v N u v= Power spectrum of the noise

2( , ) ( , )fS u v F u v= Power spectrum of the undegraded image

5 4 Wiener Filtering5.4 Wiener Filtering5 4 5 estimate the power spectrum S (u v)= S (u v) / |H(u v)|5.4.5 estimate the power spectrum

Sn(u,v)

Sf(u,v)= Sg(u,v) / |H(u,v)|

2( , ) ( , ) ( , ) ( , )g f nS u v H u v S u v S u v= +

u,vSg(u,v)

5.4.4 properties

•optimal in terms of the mean square error

Wh H( ) 0 H ( ) 0•When H(u,v)=0, Hw(u,v)=0

5.4 Wiener Filtering5.4 Wiener Filtering5.4.5 experimental results:

5.4 Wiener Filtering5 4 5 experimental results:5.4.5 experimental results:

The first raw:Image corrupted byImage corrupted bymotion blur andadditive noise

The second raw:R l f i fil iResults of inverse filtering

The third raw:The third raw:Results of Wiener filtering

5 5Estimating the Degradation Function5.5Estimating the Degradation Function(blind deconvolution)

5.5.1 Estimation by image observation

(1) Choose observed sub-image ( , )sg x y(1) Choose observed sub image ( , )sg y

(2) Denote the constructed sub-image as ˆ ( , )sf x y

(3) Assume noise is negligible

thenthen( , )( , ) ˆ ( )

G u vH u vF u v

=( , )sF u v

5 5Estimating the Degradation Function5.5Estimating the Degradation Function 5.5.2 Estimation by experimentation

δ(x y) h(x y) g(x y)If f(x,y)=δ(x,y)

Impulseδ(x,y) h(x,y) g(x,y)

then( ) ( )* ( ) ( )h hδ

Impulseresponse

( , ) ( , )* ( , ) ( , )g x y x y h x y h x yδ= =

5 5Estimating the Degradation Function5.5Estimating the Degradation Function 5.5.3 Estimation by modeling

An atmospheric turbulence model based on the physical characteristics

Hufnagel and2 2 5 / 6( )( , ) k u vH u v e− +=

Hufnagel and Stanley, 1964

( , )where k is a constant

it has the same form as the Gaussian lowpass filter

5.5Estimating the Degradation Function 5.5.3 Estimation by modeling

5 5Estimating the Degradation Function5.5Estimating the Degradation Function 5.5.3 Estimation by modeling :restoration of uniform linear motion

If T is the duration of the exposure, the effect of image motionfollows that

[ ]0 00( , ) ( ), ( )

Tg x y f x x t y y t dt= − −∫

follows that

where x0(t) and y0(t) are the time varying components ofi i h di i d di i i f i

2 ( )( , ) ( , ) j ux vyG u v g x y e dxdyπ∞ ∞ − +

∞ ∞= ∫ ∫

motion in the x-direction and y-direction. Its Fourier transform is

−∞ −∞∫ ∫2 ( )

0 00[ ( ), ( )]

T j ux vyf x x t y y t dt e dxdyπ∞ ∞ − +

−∞ −∞

⎡ ⎤= − −⎢ ⎥⎣ ⎦∫ ∫ ∫2003 Digital Image Processing

Prof.Zhengkai Liu Dr.Rong Zhang29

Reversing the order of integration:

2 ( )0 00

( , ) [ ( ), ( )]T j ux vyG u v f x x t y y t e dxdy dtπ∞ ∞ − +

−∞ −∞

⎡ ⎤= − −⎢ ⎥⎣ ⎦∫ ∫ ∫Using the translation Properties of Fourier transformation, then

0 02 [ ( ) ( )]

0( , ) ( , )

T j ux t vy tG u v F u v e dtπ− += ∫0 02 [ ( ) ( )]

0( , )

T j ux t vy tF u v e dtπ− += ∫

0 02 [ ( ) ( )]

0( , )

T j ux t vy tH u v e dtπ− += ∫By defining

( , ) ( , ) ( , )G u v H u v F u v=thenSuppose that the image in question undergoes uniform linear

2 [ ( ) ( )]T j ux t vy tπ +∫

pp g q gmotion in the x-direction only, at a rate given by 0 ( ) /x t at T=

0 02 [ ( ) ( )]

0( , ) j ux t vy tH u v e dtπ− += ∫

T juat Te dtπ−= ∫0∫sin( ) j uaT ua e

uaππ

π−=

If we allow the y-component to wary as well with the motiongiven by then the degradation function becomes0 ( ) /y t bt T=

( )( , ) sin[ ( )]( )

j ua vbTH u v ua vb eua vb

πππ

− += ++

5 6 Geometric Transformation5.6 Geometric Transformation5.6.introduction

f(x,y) T g(x,y)

( , ) [( ( , )] ( ', ')g x y T f x y f x y= =

' ( , ) ' ( , )x r x y y s x y= =where

for example: zoom out ' / 2 ' / 2x x y y= =

zoom in ' 2 ' 2x x y y= =

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.1 introduction

A geometric transformation consists of two basic operations:(1) Spatial transformation

. . . .(1) Spatial transformation

. . ..(2) Gray-level interpolation

. .空间变换

(x,y)),( ,, yx. Direct transform

f(x,y)

灰度赋值最近邻

),( ,, yxg

Inverse transform

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.2 spatial transformations

Linear correction

1 2 3( , )( )

r x y a x a y as x y b x b y b

= + += + +

Quadratic correction

1 2 3( , )s x y b x b y b= + +

2 21 2 3 4 5 6

r x y a x a y a xy a x a y a

b b b b b b

= + + + + +2 2

1 2 3 4 5 6( , )s x y b x b y b xy b x b y b= + + + + +

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.3 gray-level interpolation

•Nearest neighbor interpolationf(i,j)

f(i j+1)( , ) ( ', ')g x y f x y=

(i j+1)

f(i,j+1)

( , )f i u j v= + +

(i,j+1)f(i+1,j+1), (0,1)u v∈ ,i j Z∈ v

' ( ') ' ( ')x round x y round y= =(i 1 j)

(i+1,j+1)f(i+1,j)u

(i+1,j)

f(x’,y’)

f(i,j)

f(i j+1)A

•Bi-linear interpolation

( ' ') ( )f f i j+ +

(i j+1)

f(i,j+1)(x’,y’)( ', ') ( , )f x y f i u j v= + +

, (0,1)u v∈ ,i j Z∈

(i,j+1)f(i+1,j+1)

B(1 )(1 ) ( , ) (1 ) ( , 1)

(1 ) ( 1 ) ( 1 1)u v f i j u vf i j

u v f i j uvf i j= − − + − ++ − + + + +

(i 1 j)

(i+1,j+1)f(i+1,j) (1 ) ( 1, ) ( 1, 1)u v f i j uvf i j+ + + + +

It can be operated by a mask(i+1,j)

(1 )(1 ) (1 )(1 )u v u v

u v uv− − −

Rotation in 2D plane(x,y)

(x',y')

' cos( ) cos cos sin sinx r r rα θ α θ α θ= + = −(x,y)

αX' sin( ) cos sin sin cosy r r rα θ α θ α θ= + = +

The original coordinated of the point in polar coordinates are:

cosx r α= siny r α=

5 6 Geometric Distortion Correction

Notes:

5.6 Geometric Distortion Correction5.6.3 gray-level interpolation

Rotation in 2D planeNotes:

1. The origin is in the center of' cos sinx x yθ θ= −

' sin cosy x yθ θ= +

the center of image and the direction of x-y yaxis is opposite

2. The result2. The result image’s size is depended on the

45θ =

rotate angle

3. Interpolation is

45θ = pneededy

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.4 experimental results

grid Changed grid

spatial transform result

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.4 experimental results

Spatial transform Nearest neighborhood i l i

Bi- linear interpolation

interpolation interpolation

Summary • Inverse filtering is a very easy and accurate way to restore an image provided that we know what the bl i filt i d th t h iblurring filter is and that we have no noise

•Wiener filtering is the optimal tradeoff between the inverse filtering and noise smoothing

It i ibl t t i ith t h i ifi• It is possible to restore an image without having specific knowledge of degradation filter and additive noise. However, not knowing the degradation filter h imposes the strictest limitationsknowing the degradation filter h imposes the strictest limitations on our restoration capabilities.

homework

• 画出图象变质及图象恢复模型的方框图，数学公式表示及说明其物理意义。

• 列出逆滤波和维纳滤波图象恢复的具体• 列出逆滤波和维纳滤波图象恢复的具体步骤。

推导水平匀速直线运动模糊的点扩展函• 推导水平匀速直线运动模糊的点扩展函数的数学公式并画出曲线。

• 写出图像任意角度旋转算法

The End

Chapter5 Image Restoration - USTChome.ustc.edu.cn/~xuedixiu/image.ustc/course/dip/DIP14-ch5.pdf ·...

Documents

Chapter4 - Image Filtering and Restoration

I RImage Restoration Chapter 5 Prof. Vidya Manian Depppggt ...manian/chapter5IP.pdfConstrained Least Sqqguares Filtering With Wiener filtering we have a K as a constant of the ratio

Chapter3 Image Transforms - USTChome.ustc.edu.cn/~xuedixiu/image.ustc/course/dip/DIP14...Chapter3 Image Transforms •Preview • 31G lI d i dCl ifi i3.1General Introduction and Classification

Remote Filtering - Forcepoint€¦ · Filtering Client machine makes a browser-based Internet request, Remote Filtering Client determines whether to query Remote Filtering Server

Median Filtering and Median Filtering and Morphological Filtering

Image Filtering Advanced Image Filtering Advanced Image filtering with GDI and DX HW accelerations

SIGNALS RESTORATION BASED ON FILTERING

SYDE 575: Digital Image Processing Restoration – Chapter 5 Wiener Filtering: Sections 5.6-5.8

Introduction and Overview · and image interpolation •Intensity transformations •Spatial filtering •Fourier transform and filtering in the frequency domain •Image restoration

Image Processing in Freq. Domain Restoration / Enhancement Inverse Filtering Match Filtering / Pattern Detection Tomography

en.SafeWatch Filtering - Watch List Filtering

1 Information Filtering Rong Jin. 2 Outline Brief introduction to information filtering Collaborative filtering Adaptive filtering

Spatial filtering of interfering signals at the initial ... · Spatial filtering results (2) transmitter at horizon (26.75 MHz) subtraction filtering (26.75 MHz) projection filtering

Non Linear Image Restoration in Spatial Domainsmoothing model has been used by Gabor [1], some of these techniques uses anisotropic filtering [2,3] and the neighbourhood filtering

Signal Filtering

Digital Image Processing Introduction Image restoration ......Digital Image Processing Spatial Filtering The two primary categories of spatial filters for noise removal are • Order

Lecture 6: Multiple Model Filtering, Particle Filtering

Image Restoration by Inverse Filtering in the Frequency ...12000.org/my_notes/final_project_EECS_207A_FALL_2004_IMAGE/EEC… · Image Restoration by Inverse Filtering in the Frequency

Digital Image Processing and Image Restoration...a. Image enhancement: image enhancement deals with contrast enhancement, spatial filtering, frequency domain filtering, edge enhancement

Collaborative Filtering Recommender Systemsfaculty.chas.uni.edu/~schafer/publications/CF_AdaptiveWeb_2006.pdf · 1 Introduction Collaborative Filtering is the process of filtering