A Hybrid Filter for SEM Image Enhancement · filter using a stander image with many noise levels before applying it on the SEM image. The new filter gave better results comparing

American Journal of Research www.journalofresearch.us

5-6, May-June 2020 [email protected]

119 USA, Michigan

A Hybrid Filter for SEM Image Enhancement

Ahmed Asal Kzar*1

, Hussein Abdullah Alshamarti*2

*Physics Department, Faculty of Science, University of Kufa, Najaf, Iraq.

[email protected]

[email protected]

Corresponding author: Ahmed Asal Kzar, Ph.D. Tel: 009647732386850

Abstract

Scanning electron microscopy (SEM) images is a special and important type

of images that need special enhancement to get more information for the nanoscale

phenomenon. Many studies have been introduced in this purpose with the traditional

enhancement and suggested methods but here in this study a new method represented

in a hybrid filter formed by mean and Sobel that are common filters in enhancement

and edge detection respectively. The results are evidence for the success of the new

filter using a stander image with many noise levels before applying it on the SEM

image. The new filter gave better results comparing with the mean filter as a

traditional enhancement filter. The new filter is added as new technique filter for

Nano scale images.

Keywords:

SEM; Hybrid Filter; image enhancement; mean filter; Soble filter; Gaussian noise.

Introduction

The main tool of getting information for characterization of materials at the Nano

scale is scanning electron microscopy (SEM) [1]. The advanced technology nodes

have size of several structures that are reaching to sub-10nm, resulting in the contact-

edge roughness (CER) and the line-edge roughness (LER) smaller than few

nanometers. Because of this, the ability of the image to give such small structures

becomes critical in both inspection and metrology. Scanning electron microscopy

(SEM) is the most widely used for inspection of information about nanostructure

scale [2]. There are many reasons that produce distortion (noise) that reduces the

information in the image of (SEM). The main two important things in the image are

the color (homogenous part of the object) and the edge (limit part of the object)[3].

mailto:[email protected]

mailto:[email protected]



120 USA, Michigan

After getting resulted image that have noise by a reason or other, we have to find the

best solution for this problem to remove it or to reduce its impact, in the image

processing there are many techniques to solve the noise problem, these techniques

depend on the type of noise that produced with the image.

In information theory there are three forms of information ascending from

lowest level to the highest level: first level is the text form, second level is the sound

form, and third level is the image. Therefore, we found that always the researchers,

scientists, practitioners, engineers, and geologists depend on the image because of the

huge information that it contents. In Nano-technology field the image and its

processing like SEM image and other techniques have been widely used to help in

giving important information about these microscope phenomena. K. S. Sim and M.

E. Nia proved the efficiency and the accuracy of the image noise cross correlation

estimation model, vs. other existing estimators, they applied to different kinds of

scanning electron microscope images[4]. Naresh Marturi and others got the quality of

images obtained by a tungsten gun SEM that has been evaluated by quantifying the

level of image signal-to-noise ratio (SNR). To determine the SNR, an efficient and

online monitoring method is developed based on the nonlinear filtering using a single

image[5]. Myungjun Lee and others enhance the resolution of the CD-SEM

metrology by applying the super-resolution (SR) technique using the advanced signal

processing algorithms. Where the resolution enhancement verified by exploiting the

multiple low resolution (LR) images that contained especial information of an

imaging target through looking at a slightly different position[2]. Mike Williamson

and Andrew Neureuther introduced A recovering more accurate line edge roughness

(LER) data from SEM images by using an image deblurring technique, this study

prove that LER can vary significantly before and after deblurring[6]. E. Giannatou

and others, They trained SEM image Denoising model (SEMD) on synthesized image

data to find the variations of noise and provides as an output a denoised image of the

pattern[7].

The researchers and scientists are still introduce several image techniques to be

the best according to the giving resolution and keeping the image details that

investigated visually by human eye and quantity by statistical measurements. So,

there is no ideal enhancement technique to cover all distortion issues in SEM

imaging. Here in this work we create a new filter by combining two common

different filters from two purposes in image processing methods: mean filter which

use for smoothing and Soble filter which use for edge detection, where both of them

in one process.

https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorStored=Sim%2C+K+S

https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorStored=Nia%2C+M+E

https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorStored=Nia%2C+M+E

https://onlinelibrary.wiley.com/action/doSearch?ContribAuthorStored=Marturi%2C+Naresh

https://www.sciencedirect.com/science/article/abs/pii/S0167931719302084#!



121 USA, Michigan

Material and method

Mean filter:

Mean filter may be an easy, simple and self-evident to implement technique of

enhancing images, i.e. reducing the high difference between the intensities of the

pixel and the next one. It's usually accustomed cut back noise in the digital images.

The thought of mean filtering is just to interchange every pixel value in an image

with the mean (average) value of its neighbors, as well as itself. This filter eliminates

pixel values that square measure atypical of their surroundings. mean filter calculates

the mean value of the all pixels values of the filter size then be takes the center of the

filter[8].

it is represented by the following equation[9]:

∑ (1)

Where L is the maximum number of filter size, k is the index of pixels.

Sobel filter:

It is represented by the Gradient operator that commonly used for differentiation

in image processing. The gradient of the function I(x, y) is defined as the vector[10]:

yG

xG

y

I

x

I

)y,x(IG

(2)

Gradient operator is applied to a continuous function which produces a vector at

each point whose direction represents maximum change of the function at that point,

and whose magnitude gives magnitude of this maximum change[11, 12]. Of course

an image is not a continuous function, and it cannot be differentiated in the usual

way. Because a digital image is discrete, we use differences instead of derivatives

that are the derivative at a pixel which is approximated by the difference in gray

levels over some local region. These can then be combined together to find the

absolute magnitude of the gradient at each point and the orientation of that gradient.

The gradient magnitude is given by[13, 14]:

22 GyGx)y,x(G (3)

Typically, an approximate magnitude is computed using[15]:

GyGx)y,x(G (4)

The angle of orientation of the edge (relative to the pixel grid) giving rise to the

spatial gradient is given by:



122 USA, Michigan

Gx

Gytan 1 (5)

Where:

)y,nx(I)y,nx(IGx (6)

ny,x(I)ny,x(IGy (7)

Where n is a small integer, usually unity. Noted that numbers in Soble filter will be

changed by N of the new filter, so it led to change entire result which represents the

novelty of the new work. The edge detection masks by Sobel filter look for edges in

both of horizontal and vertical directions and then combine this information into

single metric[16]. The Sobel operator performs a 2-D spatial gradient measurement

on an image and so emphasizes regions of high spatial frequency that corresponds to

edges. The operator consists of a pair of (3x3) convolution kernel.

Sobel operator is larger convolution kernel that makes smoothing the input

image to a greater extent, in addition to makes the operator less sensitive to noise[17].

Therefore, we pick it to serve SEM images.

Sobel operator is one of the edge detection operators that return orientation

information (about the direction of the edge) , while others return only the

information about the existence of an edge in each point as Laplace operator. When

we apply the edge detection operators with an image we use the convolution equation

to get on the gradient, as in the following equation:

)jy,ix(I.)j,i(w)y,x(edg

B

Bi

B

Bj

2

1

2

1

2

1

2

1 (8)

Gaussian Noise

Gaussian noise is the most common noise in SEM images[18], therefore we

focus on this type in this work. Gaussian noise is statistical noise which has a

probability density function of the normal distribution. This means, the noise can

-1 0 1

-2 0 2

-1 0 1

1 2 1

0 0 0

-1 -2 -1

GX Gy

W(j,i)= W(i,j)=



123 USA, Michigan

takes on values as Gaussian-distributed. This type of noise is properly known as the

noise with a Gaussian amplitude distribution. Noise is modeled as additive white

Gaussian noise (AWGN), where all the image pixels values changed from their

original values because of the Gaussian curve[19]. Thus, for each image pixel with

intensity value Iij (1 ≤ i ≤ m, 1 ≤ j ≤ n for an m x n image), the noisy image that

corresponding pixel of gij are presented by the following equation[20]:

gij = Iij +Nij (9)

where, each noise value Nij is drawn as starting from zero then it takes normal

distribution.

Creating a Hybrid Filter

The creating digital filter in image processing requires understanding the main

purpose of this filter exactly, in the other word understanding the physical

phenomenon that represent the image and what the researcher want to do (process),

then representing the process digitally. In the other hand, the digital process be as

matrix contain numbers. When the noise is our problem in the image, we can use one

of the traditional filters of removing noise or enhance the noisy image, mean filter is

the most common filter that used to this task, but there is some image information

loss while it uses, especially the edges of the objects in the image, the matter that

need to maintain these information. So, because the filer number represent the main

impact with image information, the manipulation these number will help us in

maintaining information. We add a new factor with mean filter which is N factor as

in equation (10). In addition to, we mix the common filter for edge detection in

image processing as multiplying N by Sobel filter. Sobel filter helps us in maintain

the edges of objects in the image. As a result, we get enhancement the image by mean

and maintain the edge by Sobel filter with 3x3 size of window for both of them, as in

the new created equation (10 ) which represents the new hybrid filter in this work.

The hybrid filter formed by according to the following equation:

IN=MeanN

+ N*Sobel (10)

Where IN is the enhanced digital number of the resulted image, Mean is the mean

filter for enhancing the digital image, Sobel is Sobel filter for edge detection of the

digital image, and N is the factor of the new filter.

Work Algorithm

Step 1- Read the original image (standard image).

Step 2- Add Gaussian noise over a range of noise levels [0.01, 0.05] by interval

(0.01) to the original image to produce five noisy images.



124 USA, Michigan

Step 3- Apply mean filter using equation (1) with size of 3x3 to the entire noisy

image to produce five enhanced images.

Step 4- Calculate MSE between the original image and the first enhanced image by

mean.

Step 5- Repeat step 4 to calculate MSE for all enhanced images by mean filter.

Step 6- Apply the new filter using equation (10) with 3x3 of size the first noisy image

with varying of N value in the equation (10) starting from (0.04) to (0.40)

with interval (0.04) to produce 10 enhanced images.

Step 7- Repeat (step 6) to all other noisy images.

Step 8- Calculate MSE between the original image and the first enhanced image by

the new filter.

Step 9- Repeat step 8 to calculate MSE for all enhanced images by the new filter.

Step10- Pick the best N value that gave the lowest MSE to be considered in the new

filter for applying it with SEM images.

Results and discussion

V=0 V=0.04 V=0.08



125 USA, Michigan

V=0.12 V=0.16 V=0.20

Figure 1: the standard image and its noisy images

The new filter that represented by the work algorithm gave clear results in the

main two types of standard results: first, quantity which represented with table and

figures. Second, visually which represented with enhanced images? So the reader can

understand the results sequentially according to the work algorithm steps. Where the

results begin with the stander image then ended with the SEM image of the main

application that push us to achieve this work.

Results of the Stander Image

The main cause of using a stander image is because it is clear from noise and the

required image to enhance (SEM image) contains amount of noise that does not allow

to us to study and test as validation method for the new filter. Therefore, we applied

many steps on it represented by adding amounts of Gaussian noise started from 0.01

to 0.05 with an interval 0.01 as standard deviation in Gaussian noise equation. The

figures below demonstrate the N factor with mean squire error.

Figure 2: the relationship between n factor and MSE at sd=0.01

Sd=0.01

Mean filter



126 USA, Michigan




Sd=0.04

Sd=0.02

Sd=0.03



127 USA, Michigan


The adopted N in equation (10) will be from above figures. Where each n value

has especial MSE which has a not progressive variance from figure to other, this

phenomenon is because of the producing changing of N value to the both kernels of

the used filters (mean and Soble), especially on Soble filter because it has variance in

kernel values (-,0,+) and it has two directions (vertical and horizontal) as we mention

ion the equations (6) and (7). Therefore, we did not get a gradual changing in the

MSE value. Whatever, the adopted n for the main equation of the new filter is

depending on the n which has the highest iteration in these five figures.

The comparison of the new filter with mean filter through the different amounts of

Gaussian noise is demonstrated by the following table.

Results of Real SEM Image

Figures 5 and 6 related with the real images of SEM that are the main images

required for enhancement depending on tests on the standard image.

0

0,5

1

1,5

2

2,5

0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 3,6 4

Iteration

Iteration

Sd=0.

Sd=0.05



128 USA, Michigan

Figure 7: the relationship between N factor and its iteration for choosing N

a b c

d e f

Figure 8: a, b, and c the real SEM images. d, e, and f the enhanced images by

applying the new filter with (N=2).

The results of real images of SEM show the impact of the couple of processes

smoothing and edge maintaining that enhance the generated noise and keep one of the

main image information related with objects edges. Also, the new filter results prove

that are very appropriate and necessary for an image like SEM image, because it

contains a lot of shadow between rods that produce high differences between rods

colors and it, where this considered the main cause of noise. The needed filter like the

new filter in this work is very important to element noise and maintains the lines and

edges in micro image like SEM image.

Conclusion

The new hybrid filter proved its success through union of the tow important

processing for the digital image; smoothing (mean filter) and edge detection (Soble

filter) for enhancing important image type which is SEM image in the

nanotechnology. The new filter share in extracting fine information of

nanotechnology that developed by the biggest form of information represented in



129 USA, Michigan

digital image. The new filter reduces time and efforts for distinguishing nanostructure

components and help manufacturing of new composite materials.



130 USA, Michigan

Reference

[1] K. de Haan, Z. S. Ballard, Y. Rivenson, Y. Wu, and A. Ozcan, "Resolution enhancement in scanning electron microscopy using deep learning," Scientific reports, vol. 9, pp. 1-7, 2019.

[2] M. Lee, J. Cantone, J. Xu, L. Sun, and R.-h. Kim, "Improving SEM image quality using pixel super resolution technique," in Metrology, Inspection, and Process Control for Microlithography XXVIII, 2014, p. 90500U.

[3] A. A. D. Al-Zuky and Ahmed Asal Kzar, "Hit noise reduction in some x-ray images," Al-Nahrain Journal of Science, vol. 12, pp. 78-86, 2009.

[4] K. Sim, M. Nia, and C. P. Tso, "Noise variance estimation using image noise cross‐correlation model on SEM images," Scanning, vol. 35, pp. 205-212, 2013.

[5] N. Marturi, S. Dembélé, and N. Piat, "Scanning electron microscope image signal‐to‐noise ratio monitoring for micro‐nanomanipulation," Scanning: The Journal of Scanning Microscopies, vol. 36, pp. 419-429, 2014.

[6] M. V. Williamson and A. R. Neureuther, "Enhanced quantitative analysis of resist image contrast upon line-edge roughness (LER)," in Advances in Resist Technology and Processing XX, 2003, pp. 423-432.

[7] E. Giannatou, G. Papavieros, V. Constantoudis, H. Papageorgiou, and E. Gogolides, "Deep learning denoising of SEM images towards noise-reduced LER measurements," Microelectronic Engineering, vol. 216, p. 111051, 2019.

[8] S. Tiwari, A. Bansiya, and R. K. Paul, "Elimination of Impulse Noise using Enhanced Mean Median Filter for Image Enhancement," International Journal of Engineering and Management Research (IJEMR), vol. 5, pp. 811-818, 2015.

[9] Z. Lv, J. Shi, and Y. Wang, "AN ADAPTIVE MORPHOLOGICAL MEAN FILTER FOR VERY HIGH-RESOLUTION REMOTE SENSING IMAGE PROCESSING," International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, vol. 42, 2017.

[10] R. C. Gonzalez, "P. wintz digital image processing," Addision-Wesley Publishing Company, pp. 275-281, 1987.

[11] H. Alshamarti, A. K. Hussein, and B. Almayahi, "Application of median filter with the threshold technique to reduce and remove gaussian noise on the image edges produced by sobel operator," International Journal of Computer Engineering & Technology (IJCET), vol. 4, pp. 121-126, 2012.

[12] W. Niblack, "An introduction to digital image processing. 1986," in G. Leedham, C. Yan, K. Takru, J. Tan and L. Mian. Comparison of Some Thresholding Algorithms for Text/Background Segmentation in Difficult Document Images. In Proceedings of the Seventh International Conference on Document Analysis and Recognition, 2003, pp. 859-864.



131 USA, Michigan

[13] O. R. Vincent and O. Folorunso, "A descriptive algorithm for sobel image edge detection," in Proceedings of Informing Science & IT Education Conference (InSITE), 2009, pp. 97-107.

[14] H. Khosravi and E. Kabir, "Farsi font recognition based on Sobel–Roberts features," Pattern Recognition Letters, vol. 31, pp. 75-82, 2010.

[15] R. W. Smith, "Computer processing of line images: A survey," Pattern recognition, vol. 20, pp. 7-15, 1987.

[16] S. Gupta and S. G. Mazumdar, "Sobel edge detection algorithm," International journal of computer science and management Research, vol. 2, pp. 1578-1583, 2013.

[17] R. Fisher, S. Perkins, A. Walker, and E. Wolfart, "Hypermedia image processing reference," Department of Artificial Intelligence, University of Edinburg, available on http://www. cee. hw. uk, 1996.

[18] K. S. Sim, V. Teh, and M. E. Nia, "Adaptive noise wiener filter for scanning electron microscope imaging system," Scanning, vol. 38, pp. 148-163, 2016.

[19] C. Xu, S. Zhao, Y. Ma, B. Huang, and F. Liu, "Robust filter design for asymmetric measurement noise using variational Bayesian inference," IET Control Theory & Applications, vol. 13, pp. 1656-1664, 2019.

[20] H. Alshamarti, "Removal of Gaussian noise on the image edges using the Prewitt operator and threshold function technical," Journal of Computer Engineering, vol. 15, pp. 81-85, 2013.

http://www/

Documents

A Hybrid Filter for SEM Image Enhancement · filter using a stander image with many noise levels before applying it on the SEM image. The new filter gave better results comparing