Project ELEC533 Wan Pengfei

7/31/2019 Project ELEC533 Wan Pengfei

1/12

ELEC533 Final Project

AbstractIn this project, I implement and evaluate a variety

of most commonly used image denoising algorithms and propose

noticeable experimental results by comprehensive comparisons

among different denoising approaches. These algorithms include

the canonical Isotropic Filtering [5], Anisotropic Filtering [6],

Non-local Filtering [7], SUSAN Filtering [8][9], Total Variation

Minimization [10], as well as the state-of-the-art BM3D [3][4]

image denoising algorithm.

Index TermsImage denoising.

I. INTRODUCTIONMAGE denoising is a classical and flourishing research topic

in image processing. The demand for image denoisng

approaches comes from the fact that during the process that a

digital image is captured, quantized, recorded and transmitted,

it will inevitably be contaminate by a variety of noises, which

will result in annoying artifacts and decrease the visual quality.

To date, a great deal of image denoising algorithms have been

proposed [3][5][6][7][8][10], originating from various subjects

ranging from probability and statistics to differential equations,

from spatial-domain filtering to transform-domain spectrum

manipulation. Among which many have achieved remarkable

performance and been widely used in our social life.

Generally speaking, image denoising is an important topic

of image restoration, which aims at reconstructing an imagewith a certain degree of degradation. A generally degradation

model of an image can be described as follows:() () ()Wherex represents the two-dimensional location of a pixel,

u denotes the original image we want to reconstruct and v is

the noisy observation of the image. n denotes the noise, which

can be arbitrary digital noise that could occur, such as pulsing

noise, salt-andpepper noise as well as Gaussian valued noise.

Usually, when we talk about image denoising, we refer the

noise as additive white Gaussian noise (AWGN).

The goal of image denoising is to find out , which is theestimate of original image u from its noisy observation v.

Several criterions exist to evaluate the denoising performanceof different denoisng algorithms. Among which the PSNR is

probably the most widely used one. A higher PSNR indicates

a better performance of the algorithm.

PSNR is short for Peak Signal to Noise Ratio. Generally in

signal processing, signal-to-noise-ratio is used to represent the

proportion of power between the signal and the noise.

Wan Pengfei, PhD student of ECE Department, HKUST

Particularly, as for a gray-scale image with bit-depth of 8,

PSNR is defined as follows:

where MSE denotes the mean square error, i.e.

() ()

The rest of the report is organized as follows. I will briefly

introduce the six representational image denoising algorithms

in Section II. Experimental results can be found in Section III,

and finally I will conclude my work in Section IV.

II. EXISTINGDENOISINGALGORITHMSA. A brief overview

Plenty of denoisng algorithms are proposed. Among

these algorithms, some perform spatial convolution locally,

such as the Gaussian smooth model (Gabor [5]); some apply

spatial convolution in a global range, such as non-local

means [7]; some achieve denoising by anisotropic filtering

based on the texture and edge information of the image

(Perona-Malik [6]). Some exploit the prior information of

the image in both Euclidean space as well as the Intensity

space (Smith, Brady [8]). Also there are some algorithms

look into the denoisng problem in frequency domain, such asBM3D (Dabov [4]), as well as some approaches using

wavelet thresholding. Another class are the optimization-

based algorithms, which typically achieve denoising by

calculus of variations, such as Total Variation Minimization

(Rudin-Osher-Fatemi [10]). Recently, in ICIP2010, a game

theory based denoisng algorithm is proposed, which shows

the abundant research enthusiasm in image denoisng.

B. BM3DBM3D is the abbreviation of Block Matching and 3D

filtering [3][4], which is a brilliant work of Dabov, et al from

Tampere University of Technology, Finland. It is commonly

regarded as the state-of-the-art real-time denoisng algorithm.

The main idea is based on an enhanced sparse representation

of image blocks in transform-domain. The enhancement of the

sparsity is achieved by grouping similar 2D image fragments

(e.g. blocks) into 3D data arrays. Then collaborative filtering

is applied to deal with these 3D groups.

The collaborative filtering is achieved by three successive

steps: 3D transformation of 3D group, shrinkage of transform

A Modest Overview of Image Denoising

Wan Pengfei 10800496

I


2/12


spectrum, and inverse 3D transformation. The result is a 3D

estimate that consists of the jointly filtered grouped image

blocks. By attenuating the noise, the collaborative filtering

reveals the details shared by grouped blocks. And the filtered

blocks are then returned to their original positions.

Because the blocks are overlapping, for each pixel we can

obtain many different estimates which need to be combined.

Aggregation scheme is then exploited to take advantage of this

redundancy.

The generally flow chart of BM3D is described in Figure

2.1.1 Specifically, in Step1 we first apply a basic estimation to

find blocks that are similar with the current block and form a3D group together. Then we apply a separable 3D transform

on the 3D group. The separable 3D transform is composed by

a basic 2D transform and a 1D Haar transform on the third

dimension. Finally, a primary estimate of the original image is

obtained by aggregation. Aggregation is performed by a

weighted averaging at those pixels positions where there are

overlapping block-wise estimates.

In Step2, similarly grouping and 3D collaborative filtering

are applied to both the primary estimate and the noisy image.

The different in this step is the 3D filtering in Step2 makes use

of the energy spectrum of the primary estimate obtained in

Step1. So we can apply DCT and Haar-wavelet Wiener

filtering to get the final estimate ofu.The final estimate output is obtained by aggregation:

( ) Where is the estimated estimate of each image block

in different block groups via collaborative filter.

C. Isotropic filteringBy Riesz theorem, isotropic filtering of image is equivalent

to a convolution of the image by a linear symmetric kernel.

The most famous isotropic filter is undoubtedly the Gaussian

kernel:

() ()

where h is the standard deviation and the estimated image

is then represented as the convolution of the noisy observation

and the Gaussian kernel: In practice the filtering range is the rectangle neighborhood

around the current pixel. Though Gaussian isotropic filtering

is effective to remove slightly contaminated images and

simple to implement, usually the texture and edge will also be

blurred which limits the denoising performance of this class of

algorithms.

D. Anisotropic filteringIn order to alleviate the blurring effect introduced by the

isotropic filtering, the anisotropic filtering attempts to exploit

the edge information and only apply convolution on noisy

image v at pixel locations that along the direction orthogonal

to the gradient direction ().i.e.

() ()( ()

() )This algorithm is first proposed by Perona and Malik [] and

can reduce image noise without removing significant parts of

the image content, such as edges, lines or other details that are

important for the interpretation of the image.

The implementation of anisotropic filter is usually done by

iteration scheme. Formally speaking, let I denote the input

image, the partial differential equation can be described as

follows: ( )

Where denotes the Laplacian operator and denotesthe gradient operator. c(x, t) is the evolution coefficient that

controls the rate of iteratively diffusion. c is usually chosen asa function of image gradient so as to preserve the textures and

edges. In Perona and Maliks paper, two functions are

proposed for c:

() ( ) () ( )

Where factor K controls the sensitivity to edges. The first

function privileges high-contrast edges over low-contrast

edges. And the second privileges wide regions over smaller

ones.

E. Non-local filteringThe isotropic and anisotropic filtering share the same basic

idea: denoising is achieved by local linear convolution.

However, for some images with evident textures, the local

algorithms may not work well.

So the non-local algorithms try to make use of the image

information of a wider range to achieve better denoising

performance. The representational algorithm is proposed by

Buades [7], in this paper the basic idea is to find out and make

use of all the pixels whose neighborhood is similar with that of

the current pixel, i.e.

Figure 2.1.1 BM3D flow chart


3/12


() () () () { [()()] }

There are several differences between non-local filtering and

local filtering algorithms. The first one is that the range of

neighboring pixel used for denoisng is larger, theoretically all

the pixels within the image can be used to denoisng the current

pixel in Non-local filtering. The second difference is that thesimilarity measurement, which decides the averaging weight,

is not just depend on the intensity value of one pixel, but the

similarity between rectangle neighborhoods.

However, shortcomings of non-local filtering algorithm are

also obvious: a larger filtering range usually leads to over-

smoothing artifacts that textures and edges are unnecessarily

blurred. Also, the non-local algorithm suffers from relatively

higher computational complexity.

F. SUSAN filteringSUSAN filtering [8] is a kind of neighborhood filtering

which shares the same idea as Bilateral filtering [9]. Instead of

consider only the spatial neighborhood which is close to the

current pixel spatially, SUSAN filter weights pixels both by

the Euclidean distance and the Intensity-space distance, i.e.

() () () () { [()()]

}

By exploiting the inherent intensity structure of the image,

The SUSAN filter can preserves sharp edges and textures

when removing the noise, which is also an important property

of anisotropic filter.

Another advantage of SUSAN filter (Bilateral filter) is that

unlike the anisotropic filtering, it is a non-iterative edge-preserving denoising algorithm, which makes it widely used,

even adopted by the well-known Adobe Photoshop.

G. Total variation minimizationThis algorithm regards the image denoising problem as a

minimization problem. It was first introduced by Rudin, Osher

and Fatemi [10]. Given a noisy image observation v, the Total

Variation Minimization algorithm try to estimate the original

image u via the solution of the following expression: ()

() ()

() () under the constrains

( () () ) () ()

By introducing a Lagrange multiplier , this minimizationproblem becomes:

() () ()The above function is strictly convex, so that the minimum

exits and is computable. In this algorithm, the noise v(x) - u(x)

is treated as an error. In practice, some textures are usually

presented in the error. One possible solution to this problem is

an iterative algorithm proposed by Burger [11] as follows:

1. First solve the original total variation model: () () ()

2. Perform a correction step: () ()

3. Iteration to obtain the final output: ()

The solution converges after plenty of iterations and serves

as the output of Total Variation Minimization algorithm.

III. EXPERIMENTALEVALUATIONIn my experiments, six image denoising algorithms above-

mentioned are evaluated under a wide range of AWGN

( ). And 8 test images are used to make the resultsmore reasonable and convincing.

The denoising performance is evaluated under both the

subjective criterion of visual quality and the objective criterion,

which is the absolute PSNR of the estimated image as wellas PSNR increment compared with the noisy image v.

A. Parameters SelectionIn my implementation, I make use of training images to

select rational and adaptive model parameters to achieve

best-so-far performance for each algorithm.

Figure 3.1.1 Adaptive for Gaussian isotropic filter

Figure 3.1.2 Adaptive for iterative anisotropic filter


4/12


Figure 3.1.3 Adaptive of intensity for SUSAN filter

Figure 3.1.4 Adaptive for iterative solution of TVMB. Performance comparison w.r.t same test imagesI test the denoising performance of each algorithm on eight

images measured by PSNR increment of the restored image compared with the noisy image .Below are several numerical comparisons of the six

denoising algorithms on different test images.

where,

The x coordinator is the standard variance of AWGN,

ranging from 5 to 40.

The y coordinator is the PSNR increment of the denoised

image comparing with the noisy image.

Different algorithms are marked in different colors.

Figure 3.2.1 Performance comparison on image with textures

Figure 3.2.2 Performance comparison on a natural image

We can see from Figure 3.2.1-2:

For natural images (such as one of my photos) BM3D can

achieve incomparable denosing performance. As for other

algorithms, Total Variation Minimization, SUSAN filter,Non-local filter and Isotropic filter can achieve satisfactory

results that the PSNR increments converge to approximately

8-9 dB.

For artificial images with strong textures and edges (such as

montage), besides BM3D, the edge-preserving algorithms,

SUSAN filtering and Anisotropic filtering, can achieve

appealing results which is comparable with theTVM algorithm.

However, understandably, the Isotropic filter shows a poor

performance.

In both occasions, a sharp performance drop is observed for

the Anisotropic filter, which I will explain in the next section.

C. Overall performance evaluation

Figure 3.3.1 Average PSNR increment comparison

The figure above shows the average denoising performance

of each algorithm on the whole test image database.

5 10 15 20 25 30 35 400

2

4

6

8

10

12

14Denoisng Peformance Comparison on montage.png

Variance of AWGN

PSNR

incrementcomparedwithnoisyimage

BM3D

Isotropic Filter

Anisotropic Filter

Non-local Filter

SUSAN Filter

TV Minimization

5 10 15 20 25 30 35 400

2

4

6

8

10

12

14

16Denoisng Peformance Comparison on me.jpg

Variance of AWGN

PSNR


BM3D

Isotropic Filter

Anisotropic Filter

Non-local Filter

SUSAN Filter

TV Minimization

5 10 15 20 25 30 35 400

2

4

6

8

10

12

14Average Denoisng Peformance Comparison

Variance of AWGN

AveragePSNR

increment

BM3D

Isotropic Filter

Anisotropic Filter

Non-local Filter

SUSAN Filter

TV Minimization


5/12


We can conclude that statistically:

1. BM3D achieves apparently the best objective denoisingperformance.

2. TVM, SUSAN filter and Anisotropic filter (AF) achievethe betterthanaverage denoising performance.

3. The performance of Isotropic filter (IF) and Non-localfilter (NL) is relatively poorer.

The overall image denoising performance can be stated as

follows: which is substantially consistent with our prior expectation

on these denoising algorithms. The reason for the poor

performance of Non-local algorithm is that the good result of

Non-local filtering requires highly regularity of image textures,

which is generally not true for natural images.

D. Performance comparison w.r.t different noise varianceIn this section, lets consider how denoising performance

evolves with different degree of noise for each algorithm. The

following conclusions can be drawn based on Figure 3.3.1.:

When the noise variance is relatively small, the denoising

performance follows: Notice that the SUSAN filter performs much poorer when

the variance is small, that is because the inner scheme of

SUSAN filter is similar with Gaussian isotropic filter. Usually

the blurring artifact will become more prominent when the

noise variance is small.

When the noise variance is relatively large, the denoising

performance follows: Notice that a sharp drop is observed for Anisotropic filter

when the noise variance becomes large. This is probably

because of the fact that the anisotropic filtering is essentially a

diffusion process, when the signal-noise-ratio is large thesevere error propagation will usually lead to an unstable

solution.

E. Local vs Non-localDenoising performances of local filtering (Isotropic filter)

and the Non-local filter are compared in this section.

Figure 3.5.1 Performance comparison on natural image

Figure 3.5.2 Performance comparison on artificial image

We can see that for natural images, the Non-local filter have

slightly better results for small noise variance and slightly

poorer results for large noise variance compared with Isotropic

filter, so basically they are similar in denoising performance.However, for artificial images (such as montage), because of

the fact that much more textures exist, the Non-local filter can

make use of plenty of similar image patches to denoise the

current pixel. So the Non-local filtering can achieve a huge

performance gap over the local Gaussian filter, which suffers

from the unnecessary removal of important high frequency

contents.

In addition, the computational complexity of Non-local filter

is much larger than that of Gaussian isotropic filtering, which

diminishes the advantage of non-local denoising algorithms to

some extent.

.

F. Isotropic vs AnisotropicAlso, we test the denoising performance of Isotropic filtering

and the Anisotropic filtering.

The motivation is to find out under what circumstances the

Anisotropic filter will have a better performance than classical

Isotropic filter as expected.

Figure 3.6.1 Performance comparison on natural image

5 10 15 20 25 30 35 400

1

2

3

4

5

6

7

8Local vs Non-local claire.jpg

Variance of AWGN

PSNR


Isotropic Filter

Non-local Filter

5 10 15 20 25 30 35 400

1

2

3

4

5

6

7

8Local vs Non-local montage.png

Variance of AWGN

PSNR


Isotropic Filter

Non-local Filter

5 10 15 20 25 30 35 400

2

4

6

8

10

12Isotropic vs Anisotropic me.jpg

Variance of AWGN

PSNR

incrementcomparedw

ithnoisyimage

Isotropic Filter

Anisotropic Filter


6/12


Figure 3.6.2 Performance comparison on artificial image

From the two figures above we can observe that:

For natural images, usually there arent a lot of textures and

edges, Anisotropic can only achieve a moderate performanceimprovement over Isotropic filter. Whats worse, Anisotropic

filter may even perform poor when the noise variance is very

large, the reason is that the implementation of Anisotropic

filter is a iterative algorithm, the error propagation may be

catastrophic in such conditions.

But for artificial images with stronger textures, such as

montage, the Anisotropic filter can achieve a larger gap over

Isotropic filter on PSNR increment than denoising natural

images. That is because the Anisotropic filter apply filtering

only on directions orthogonal to the edges, whereas the

Isotropic filter apply filtering the image without taking any use

of the inherent structure information of the image.

G. Anisotropic vs SUSANSUSAN filter and Anisotropic filter are both well-known

edge-preserving denoising algorithms. In this experiment, I

test and compare the numerical performances of these two

algorithms

Figure 3.7.1 Performance on image with moderate texture

Figure 3.6.2 Performance on image with strong texture

We can see from Figure 3.6.1-2 that both algorithms can

achieve appealing results on textured images. Particularly, for

images with moderate textures and edges, the two algorithmscan effectively remove the noise without sacrificing important

image features.

As for images with strong structures, the SUSAN filter can

achieve a slightly better PSNR increment than the Anisotropic

filter. Which is consistent with the experimental conclusions

in [8].

IV. CONCLUSIONS&DISCUSSIONSIn this work we have implemented and evaluated six classes

of most popular image denoisng algorithms. The theoretical

analysis and experimental evaluations show valuable results

on those denoising algorithms:

1. BM3D algorithm can achieve the best denoisingperformance under most occasions in real time.2. Gaussian kernel isotropic filtering can achieve

satisfactory results when the noise variance is large, but

the visual quality is poor in most cases because of the

removal of the textures and contours.

3. Iterative anisotropic filtering can effectively preservethe edges and textures with higher curvature, but the

denoising performance is greatly constrained by the

error propagation when noise variance becomes large. .

4. Non-local filtering is effective for denoising imageswith regular textures. However for general occasions

over-smoothing are usually observed. And the inner

block-matching scheme is relatively time-consuming.5. SUSAN filtering (or Bilateral filtering) is another edge-preserving denoising algorithm, which can effectively

preserve textures and edges with higher contrast, in

most cases it can achieve stable better-than-average

denoising performances.

6. Total Variation Minimization algorithm achievesdenoising by minimizing the image variance under the

constraints of noise statistics. With an iterative solution

the computational complexity is greatly reduced and

the subjective and objective denoising performances

are better than all the others except for BM3D.

5 10 15 20 25 30 35 400

1

2

3

4

5

6

7

8

9Isotropic vs Anisotropic montage.png

Variance of AWGN

PSNR


Isotropic Filter

Anisotropic Filter

5 10 15 20 25 30 35 401

2

3

4

5

6

7

8

9Anisotropic vs SUSAN montage.png

Variance of AWGN

PSNR

incrementcompar

edwithnoisyimage

Anisotropic Filter

SUSAN Filter

5 10 15 20 25 30 35 400.5

1

1.5

2

2.5

3

3.5

4Anisotropic vs SUSAN fingerprint.png

Variance of AWGN

PSNR


Anisotropic Filter

SUSAN Filter


7/12


In a nutshell, although image denoising is well-known an

ill-pose problem, current algorithms can achieve satisfactory

performances in most cases. Put another way, there is still

much work to be done on this topic, future research directions

as far as I can see could be the hybrid optimization-based

denoising algorithms.

REFERENCES[1] http://en.wikipedia.org/wiki/Main_Page/[2] http://www.cs.tut.fi/~foi/GCF-BM3D/[3] V. Katkovnik, A. Foi, K. Egiazarian, and J. Astola, From local kernel

to nonlocal multiple-model image denoising, IJCV 2010.

[4] K. Dabov, A. Foi, Image denoising by sparse 3D transform-domaincollaborative filtering. IEEE Trans. Image Processing 2007

[5] M. Lindenbaum, M. Fischer, On Gabor contribution to imageenhancement. Pattern Recognition, 1994.

[6] P. Perona, J. Malik, Scale space and edge detection using anisotropicdiffusion.IEEE Trans. PAMI1990.

[7] A. Buades, B. Coll, A non-local algorithm for image denoising.Proc.CVPR 2005.

[8] S. Smith, J. Brady, SUSAN-a new approach to low level imageprocessing, IJCV 1997

[9] C. Tomasi, R. Manduchi, Bilateral filtering for gray and color images,Proc. ICCV 1998

[10] L. Rudin, S. Osher, Total variation based image restoration with freelocal constrains, Proc.ICIP. 1994.

[11] S. Osher, M. Burger, Using Geometry and iterated refinement forinverse problems: Total variation based image restoration, JSIAM.2004

APPENDIXES

Below are some subjective denoising performance comparisons

From left to right: original image, noisy image, denoised image

From top to bottom: BM3D, IF, AF, NL, SUSAN, TVM

A. Noise standard variance
http://en.wikipedia.org/wiki/Main_Page/http://en.wikipedia.org/wiki/Main_Page/http://www.cs.tut.fi/~foi/GCF-BM3D/http://www.cs.tut.fi/~foi/GCF-BM3D/http://www.cs.tut.fi/~foi/papers/KFKA-LocalNonLocalDenoising-IJCV-Preprint2009.pdfhttp://www.cs.tut.fi/~foi/papers/KFKA-LocalNonLocalDenoising-IJCV-Preprint2009.pdfhttp://www.cs.tut.fi/~foi/papers/KFKA-LocalNonLocalDenoising-IJCV-Preprint2009.pdfhttp://www.cs.tut.fi/~foi/papers/KFKA-LocalNonLocalDenoising-IJCV-Preprint2009.pdfhttp://www.cs.tut.fi/~foi/papers/KFKA-LocalNonLocalDenoising-IJCV-Preprint2009.pdfhttp://www.cs.tut.fi/~foi/papers/KFKA-LocalNonLocalDenoising-IJCV-Preprint2009.pdfhttp://www.cs.tut.fi/~foi/papers/KFKA-LocalNonLocalDenoising-IJCV-Preprint2009.pdfhttp://www.cs.tut.fi/~foi/GCF-BM3D/http://en.wikipedia.org/wiki/Main_Page/


8/12



9/12


B. Noise standard variance


10/12


11/12


C. Noise standard variance


12/12


Documents

Project ELEC533 Wan Pengfei