Chapter5 Image Restoration - USTChome.ustc.edu.cn/~xuedixiu/image.ustc/course/dip/DIP14-ch5.pdf · Unconstrained Restoration: inverse filtering Constrained Restoration: wiener filtering

Chapter5 Image Restoration

• Preview5 1 I d i• 5.1 Introduction

• 5.2 Diagonalizationi d i ( i fil i )• 5.3 Unconstrained Restoration( inverse filtering)

• 5.4 Constrained Restoration( wiener filtering)• 5.5 Estimating the Degradation Function• 5.6 Geometric Distortion Correction• 5.7 image inpaiting

2003 Digital Image Processing Prof.Zhengkai Liu Dr.Rong Zhang

1

5.1 Introduction5.1.1 Purpose

"compensate for" or "undo" defects which degrade an image.

5.1.2 Degrade Causes

( ) h i b l

co pe s e o o u do de ec s w c deg de ge.

(1) atmospheric turbulence(2) sampling, quantization p254(3) motion blur(4) camera misfocus(5) noise


2

5.1 Introduction5.1.3 degradation model

n(x,y)

f( ) H g(x y)f(x,y) H + g(x,y)

Assume it is a linear, position- invariant system, We can model a blurred image by

( , ) ( , )* ( , ) ( , )g x y f x y h x y n x y= +

h h( ) i ll d i d i ( )Where h(x,y) is called as Point Spread Function (PSF)

∑∑− −

+−−=1 1

),(),(),(),(M N

yxnnymxhnmfyxg2003 Digital Image Processing

Prof.Zhengkai Liu Dr.Rong Zhang3

∑∑= =

+0 0

),(),(),(),(m n

yxnnymxhnmfyxg

5.1 Introduction5.1.4 Methods

Unconstrained Restoration: inverse filteringConstrained Restoration: wiener filtering

5.1.5 problem expression

Estimate a true image f(x,y) from a degraded image g(x,y)b d i k l d f h( ) d h i i lbased on prior knowledge of PSF h(x,y)and the statisticalproperties of noise n(x,y)


4

5.2 Diagonalization 5.2.1 Matrix expression of degradation model: 1-D

( ) ( )* ( )g x f x h x=

( ) 0 1( )

f x x Af x

≤ ≤ −⎧= ⎨( )

0 elseef x = ⎨⎩

( ) 0 1( )

0 elsee

h x x Bh x

≤ ≤ −⎧= ⎨⎩ 0 else⎩


5


1

0( ) ( ) ( ) ( )

M

e e e eg x f m h x m n x−

= − +∑ x=0 1 M-1

M=A+B-1

⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤

0m= x 0,1,…,M 1

(0) (0) ( 1) ( 1) (0) (0)(1) (1) (0) ( 2) (1) (1)

e e e e e e

e e e e e e

g h h h M f ng h h h M f n

g Hf n

− − +⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥− +⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥= + = = +⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥( -1) ( 1) ( 2) (0) ( -1) ( 1)e e e e e eg M h M h M h f M n M⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥− − −⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦


6


( ) ( )h x h x M= +( ) ( )e eh x h x M+(0) ( 1) (1)e e eh h M h−⎡ ⎤

⎢ ⎥(1) (0) (2)

( 1) ( 2) (0)

e e eh h hH

h M h M h

⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦( 1) ( 2) (0)e e eh M h M h⎢ ⎥− −⎢ ⎥⎣ ⎦

H is circulant


7


( , ) 0 1 and 0 1( )

f x y x A y Bf x

≤ ≤ − ≤ ≤ −⎧= ⎨( )

0 A 1 or B 1ef xx M y N

= ⎨ ≤ ≤ − ≤ ≤ −⎩

( , ) 0 1 and 0 1( )

h x y x C y Dh x

≤ ≤ − ≤ ≤ −⎧= ⎨( )

0 A 1 or B 1eh xx M y N

= ⎨ ≤ ≤ − ≤ ≤ −⎩


8


y=0 1 N-1

x=0,1,…,M-11 1

0 0( , ) ( , ) ( , )

M N

e e em n

g x y f m n h x m y n− −

= =

= − −∑∑y 0,1,…,N 1

0 1 1

1 0 2

(0) (0)(1) (1)

M e e

e e

H H H f nH H H f n

g Hf n

−⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥= + = +⎢ ⎥ ⎢ ⎥ ⎢ ⎥

1 2 0 ( -1) ( 1)M M e e

g f

H H H f MN n MN− −

⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥−⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦

H is block-circulant


9


( 0) ( 1) ( 1)h i h i N h i−⎡ ⎤

where

( ,0) ( , 1) ( ,1)( ,1) ( ,0) ( , 2)

e e e

e e ei

h i h i N h ih i h i h i

H

−⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥( , 1) ( , 2) ( ,0)e e eh i N h i N h i⎢ ⎥− −⎢ ⎥⎣ ⎦


10

5.2 Diagonalization 5.2.2 Diagonliztion: 1-D

The eigenvector and eigenvalue of a circulant matrix H are

2 2( ) 1 exp exp ( 1)T

w k j k j M kM Mπ π⎡ ⎤⎛ ⎞ ⎛ ⎞= −⎜ ⎟ ⎜ ⎟⎢ ⎥⎝ ⎠ ⎝ ⎠⎣ ⎦

g g

M M⎝ ⎠ ⎝ ⎠⎣ ⎦

2 2( ) (0) ( 1)exp (1)exp ( 1)j jk h h M k h M kπ πλ ⎛ ⎞ ⎛ ⎞= + + +⎜ ⎟ ⎜ ⎟( ) (0) ( 1)exp (1)exp ( 1)e e ek h h M k h M kM M

λ = + − − + + − −⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠

Combine the M eigenvectors to a matrix

[ (0) (1) ( 1)]W w w w M= −g

then the H can be expressed as1H WDW −= ( , ) ( )D k k kλ=

then the H can be expressed aswhere


11


g Hf n= +for

1 1 1

1 1 1

W g W Hf W nW WDW f W n

− − −

− − −

= +

= +1 1DW f W n− −= +

( ) ( ) ( ) ( )G u H u F u N u+( ) ( ) ( ) ( )G u H u F u N u= +


12


2( , ) exp NW i m j im WMπ⎛ ⎞= ⎜ ⎟

⎝ ⎠

2( , ) expNW k n j knNπ⎛ ⎞= ⎜ ⎟

⎝ ⎠

1H WDW −= ( , ) ( , ) ( , ) ( , )G u v H u v F u v N u v= +

g(x,y) = f(x,y)*h(x,y) +n(x,y) in Spatial Coordinates

G(u,v) = F(u,v)H(u,v) +N(u,v) in frequency domain

g = Hf + n in vector form


13

5.3 Inverse Filtering5.3 Inverse Filtering5.3.1 assumption

H i i d th i i li iblH is given, and the noise is negligible

G( )

5.3.2 degradation model

F(u,v) H G(u,v)

( , ) ( , ) ( , )G u v H u v F u v=


14

5.3 Inverse Filtering5.3 Inverse Filtering5.3.3 restoration

( , )ˆ ( , ) ( , ) ( , )( , ) I

G u vF u v G u v H u vH u v

= =

1( , )( , )IH u v

H u v= deconvolution

5.3.4 properties ( , )ˆ ( , )( )

G u vF u vH u v

=( , )

( , ) ( , ) ( , )( , )

H u vH u v F u v N u v

H u v+

=( , )( , )( , )( , )

N u vF u vH u v

= +


15

5.3 Inverse Filtering5.3 Inverse Filtering5.3.4 properties

sensitive to additive noise: if H(u,v) has zero or very small value,the N(u,v)/H(u,v) could easily dominate the estimate

5.3.5 improvements p2 2 2

01/ ( , ) if ( , )

1 elseH u v u v w

M u v⎧ + <

= ⎨⎩

H(u,v)⎩

or if ( , )

( , )1/ ( , ) elsek H u v d

M u vH u v

<⎧= ⎨⎩

u,v


16

5.3 Inverse Filtering5.3.4 examples: without noise

Original image Blurred image Restored image


17

5.3 Inverse Filtering5.3.4 examples: with noise

Blurred and noised image Restored image


18

5.3 Inverse Filtering5.3.4 examples: deferent cutoff


19

5 4 Wiener Filtering5.4 Wiener Filtering5.4.1 assumption

H is given, and consider the image and noise as random processes

5 4 2 request5.4.2 requestThe mean square error between uncorrupted image and estimated image is minimized This error measure is given byimage is minimized. This error measure is given by

{ }{ }2 2ˆ( )e E f f= −


20

5 4 Wiener Filtering5.4 Wiener Filtering5.4.3 restoration

ˆ ( , ) ( , ) ( , )WF u v G u v H u v=

Wi filt2

2

( , )1( , )( , ) ( , ) ( , ) / ( , )

Wn f

H u vH u v

H u v H u v S u v S u v= ×

+

Wiener filter, 1942

*

2( , )

( , ) ( , ) / ( , )n f

H u vH u v S u v S u v

=+ f

where2( , ) ( , )nS u v N u v= Power spectrum of the noise

2( , ) ( , )fS u v F u v= Power spectrum of the undegraded image


21

5 4 Wiener Filtering5.4 Wiener Filtering5 4 5 estimate the power spectrum S (u v)= S (u v) / |H(u v)|5.4.5 estimate the power spectrum

Sn(u,v)

Sf(u,v)= Sg(u,v) / |H(u,v)|

2( , ) ( , ) ( , ) ( , )g f nS u v H u v S u v S u v= +

u,vSg(u,v)

5.4.4 properties

•optimal in terms of the mean square error

Wh H( ) 0 H ( ) 0•When H(u,v)=0, Hw(u,v)=0


22

5.4 Wiener Filtering5.4 Wiener Filtering5.4.5 experimental results:


23

5.4 Wiener Filtering5 4 5 experimental results:5.4.5 experimental results:

The first raw:Image corrupted byImage corrupted bymotion blur andadditive noise

The second raw:R l f i fil iResults of inverse filtering

The third raw:The third raw:Results of Wiener filtering


24

5 5Estimating the Degradation Function5.5Estimating the Degradation Function(blind deconvolution)

5.5.1 Estimation by image observation

(1) Choose observed sub-image ( , )sg x y(1) Choose observed sub image ( , )sg y

(2) Denote the constructed sub-image as ˆ ( , )sf x y

(3) Assume noise is negligible

thenthen( , )( , ) ˆ ( )

ss

G u vH u vF u v

=( , )sF u v


25

5 5Estimating the Degradation Function5.5Estimating the Degradation Function 5.5.2 Estimation by experimentation

δ(x y) h(x y) g(x y)If f(x,y)=δ(x,y)

Impulseδ(x,y) h(x,y) g(x,y)

then( ) ( )* ( ) ( )h hδ

Impulseresponse

( , ) ( , )* ( , ) ( , )g x y x y h x y h x yδ= =


26

5 5Estimating the Degradation Function5.5Estimating the Degradation Function 5.5.3 Estimation by modeling

An atmospheric turbulence model based on the physical characteristics

Hufnagel and2 2 5 / 6( )( , ) k u vH u v e− +=

Hufnagel and Stanley, 1964

( , )where k is a constant

it has the same form as the Gaussian lowpass filter


27

5.5Estimating the Degradation Function 5.5.3 Estimation by modeling


28

5 5Estimating the Degradation Function5.5Estimating the Degradation Function 5.5.3 Estimation by modeling :restoration of uniform linear motion

If T is the duration of the exposure, the effect of image motionfollows that

[ ]0 00( , ) ( ), ( )

Tg x y f x x t y y t dt= − −∫

follows that

0∫

where x0(t) and y0(t) are the time varying components ofi i h di i d di i i f i

2 ( )( , ) ( , ) j ux vyG u v g x y e dxdyπ∞ ∞ − +

∞ ∞= ∫ ∫

motion in the x-direction and y-direction. Its Fourier transform is

−∞ −∞∫ ∫2 ( )

0 00[ ( ), ( )]

T j ux vyf x x t y y t dt e dxdyπ∞ ∞ − +

−∞ −∞

⎡ ⎤= − −⎢ ⎥⎣ ⎦∫ ∫ ∫2003 Digital Image Processing

Prof.Zhengkai Liu Dr.Rong Zhang29


Reversing the order of integration:

2 ( )0 00

( , ) [ ( ), ( )]T j ux vyG u v f x x t y y t e dxdy dtπ∞ ∞ − +

−∞ −∞

⎡ ⎤= − −⎢ ⎥⎣ ⎦∫ ∫ ∫Using the translation Properties of Fourier transformation, then

0 02 [ ( ) ( )]

0( , ) ( , )

T j ux t vy tG u v F u v e dtπ− += ∫0 02 [ ( ) ( )]

0( , )

T j ux t vy tF u v e dtπ− += ∫


30


0 02 [ ( ) ( )]

0( , )

T j ux t vy tH u v e dtπ− += ∫By defining

( , ) ( , ) ( , )G u v H u v F u v=thenSuppose that the image in question undergoes uniform linear

2 [ ( ) ( )]T j ux t vy tπ +∫

pp g q gmotion in the x-direction only, at a rate given by 0 ( ) /x t at T=

0 02 [ ( ) ( )]

0( , ) j ux t vy tH u v e dtπ− += ∫

2 /

0

T juat Te dtπ−= ∫0∫sin( ) j uaT ua e

uaππ

π−=


31

uaπ


If we allow the y-component to wary as well with the motiongiven by then the degradation function becomes0 ( ) /y t bt T=

( )( , ) sin[ ( )]( )

j ua vbTH u v ua vb eua vb

πππ

− += ++


32


33

5 6 Geometric Transformation5.6 Geometric Transformation5.6.introduction

f(x,y) T g(x,y)

( , ) [( ( , )] ( ', ')g x y T f x y f x y= =

' ( , ) ' ( , )x r x y y s x y= =where

for example: zoom out ' / 2 ' / 2x x y y= =

zoom in ' 2 ' 2x x y y= =


34


35

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.1 introduction

A geometric transformation consists of two basic operations:(1) Spatial transformation

. . . .(1) Spatial transformation

. . ..(2) Gray-level interpolation

. .空间变换

(x,y)),( ,, yx. Direct transform

f(x,y)

灰度赋值最近邻

),( ,, yxg

Inverse transform


36

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.2 spatial transformations

Linear correction

1 2 3( , )( )

r x y a x a y as x y b x b y b

= + += + +

Quadratic correction

1 2 3( , )s x y b x b y b= + +

2 21 2 3 4 5 6

2 2

( , )

( )

r x y a x a y a xy a x a y a

b b b b b b

= + + + + +2 2

1 2 3 4 5 6( , )s x y b x b y b xy b x b y b= + + + + +


37


38

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.3 gray-level interpolation

•Nearest neighbor interpolationf(i,j)

f(i j+1)( , ) ( ', ')g x y f x y=

(i j+1)

f(i,j+1)

( , )f i u j v= + +

(i,j)

(i,j+1)f(i+1,j+1), (0,1)u v∈ ,i j Z∈ v

' ( ') ' ( ')x round x y round y= =(i 1 j)

(i+1,j+1)f(i+1,j)u

(i+1,j)


39


f(x’,y’)

f(i,j)

f(i j+1)A

•Bi-linear interpolation

( ' ') ( )f f i j+ +

(i j+1)

f(i,j+1)(x’,y’)( ', ') ( , )f x y f i u j v= + +

, (0,1)u v∈ ,i j Z∈

(i,j)

(i,j+1)f(i+1,j+1)

B(1 )(1 ) ( , ) (1 ) ( , 1)

(1 ) ( 1 ) ( 1 1)u v f i j u vf i j

u v f i j uvf i j= − − + − ++ − + + + +

(i 1 j)

(i+1,j+1)f(i+1,j) (1 ) ( 1, ) ( 1, 1)u v f i j uvf i j+ + + + +

It can be operated by a mask(i+1,j)

(1 )(1 ) (1 )(1 )u v u v

u v uv− − −

−


40


Y

Rotation in 2D plane(x,y)

(x',y')

' cos( ) cos cos sin sinx r r rα θ α θ α θ= + = −(x,y)

θ

αX' sin( ) cos sin sin cosy r r rα θ α θ α θ= + = +

The original coordinated of the point in polar coordinates are:

cosx r α= siny r α=


41

5 6 Geometric Distortion Correction

Notes:

5.6 Geometric Distortion Correction5.6.3 gray-level interpolation

Rotation in 2D planeNotes:

1. The origin is in the center of' cos sinx x yθ θ= −

' sin cosy x yθ θ= +

the center of image and the direction of x-y yaxis is opposite

2. The result2. The result image’s size is depended on the

x

45θ =

rotate angle

3. Interpolation is


42

45θ = pneededy

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.4 experimental results

grid Changed grid

spatial transform result


43

5 6 Geometric Distortion Correction5.6 Geometric Distortion Correction5.6.4 experimental results

Spatial transform Nearest neighborhood i l i

Bi- linear interpolation


44

interpolation interpolation

Summary • Inverse filtering is a very easy and accurate way to restore an image provided that we know what the bl i filt i d th t h iblurring filter is and that we have no noise

•Wiener filtering is the optimal tradeoff between the inverse filtering and noise smoothing

It i ibl t t i ith t h i ifi• It is possible to restore an image without having specific knowledge of degradation filter and additive noise. However, not knowing the degradation filter h imposes the strictest limitationsknowing the degradation filter h imposes the strictest limitations on our restoration capabilities.


45

homework

• 画出图象变质及图象恢复模型的方框图，数学公式表示及说明其物理意义。

• 列出逆滤波和维纳滤波图象恢复的具体• 列出逆滤波和维纳滤波图象恢复的具体步骤。

推导水平匀速直线运动模糊的点扩展函• 推导水平匀速直线运动模糊的点扩展函数的数学公式并画出曲线。

• 写出图像任意角度旋转算法


46

The End


47

Documents

Chapter5 Image Restoration - USTChome.ustc.edu.cn/~xuedixiu/image.ustc/course/dip/DIP14-ch5.pdf · Unconstrained Restoration: inverse filtering Constrained Restoration: wiener filtering