DETECTING AND CLASSIFYING BLURRED IMAGE REGIONS

Wei Xu1, Jane Mulligan2, Di Xu3 and Xiaoping Chen1

1Univ. of Sci. and Tech. of China 2Flashback Technologies, Inc. 3Univ. of PittsburghSchool of Computer Sci. and Tech. Boulder, CO, USA Department of ECExuwei401@gmail.com, i.jane.mulligan@gmail.com, xudipitt@gmail.com, xpchen@ustc.edu.cn

ABSTRACT

Many image deblurring algorithms perform blur kernel esti-

mation and image deblurring by assuming the blur type and

distribution are already known. However, in practice such

information is not known in advance and must be estimat-

ed using local blur measures. In this paper, we revisit the

image partial blur detection and classification problem and

propose several new or enhanced local blur measures using

different types of image information including color, gradien-

t and spectral information. The proposed measures demon-

strate stronger discriminative power, better across-image sta-

bility or higher computational efficiency than previous ones.

By learning the optimal combination of these measures with

SVM classifiers, we obtain a patch-based image partial blur

detector and classifier. Experiments on a large dataset of real

images show the proposed approach has superior performance

to the state-of-the-art approach.

Index Terms— Blur Detection, Partial Blur, Image De-

blurring

1. INTRODUCTION AND BACKGROUND

Identifying blur in a given image is a challenging and long

studied topic in image processing and analysis. Automatic

deblurring has already become an important function of cur-

rent image and video processing systems. Given a blurred

image, the system does not know in advance whether it is

fully blurred or partially blurred, or whether the blurring aris-

es from motion or defocus. Before any deblurring operation,

the system needs to identify the type of blur so that the most

suitable deblurring algorithm can be used. It also needs to

analyze the blurred locations in the image so that the exact,

homogeneously blurred region can be extracted for estimat-

ing the blur kernel.

Early research on image blur focused on developing d-

ifferent kinds of whole-image blurriness measures driven by

the camera manufacturers’ need for camera auto-focus and

This work is supported by the National Hi-Tech Project of China under

grant 2008AA01Z150 and the Natural Science Foundation of China under

grant 60745002 and 61175057, as well as the USTC Key Direction Project

and the USTC 985 Project.

anti-shake functionalities (e.g, [1]). Later, interest shifted

from identifying whether an image was blurred, to identifying

and recognizing the blur regions and blur type in a partially

blurred image. The latter task is more challenging, because

it requires the blur measures to be very sensitive to small

amounts of blur and to work locally, while at the same time to

be robust to noise. Many previously proposed blur measures

become “weak” under such requirements, thus people began

to think about combining multiple blur measures locally to

achieve the goal of partial blur detection and classification.

Unlike object detection in which a high-level template

of the target object can be used to guide the organizing of

low-level features, blur detection does not have any high-

level “shape” template for reference but has to rely on low-

level image analysis. Moreover, for detecting blurred regions

in a partially blurred image, many existing low-level blurri-

ness/sharpness metrics developed for measuring the quality

of the whole image (e.g. for “auto-focus” applications [2])

are not applicable. In addition, local blur measures relying

on sparse image features (e.g. edge sharpness [3]) are not ap-

plicable because the existence and density of these features

cannot be guaranteed within every local patch.

In this paper we propose an approach to partial blur detec-

tion and classification which combines different types of lo-

cal blur measures to build classifiers for partial blur detection

and blur type. To serve this solution, we analyze the deficien-

cies and incorrect usage of some previous local blur measures,

and develop a set of new or enhanced local blur measures with

stronger discriminative power, better across-image stability or

higher computational efficiency.

2. RELATED WORK

There has been a lot of work on image blur analysis and pro-

cessing in recent years. However, most previous work focus-

es on the estimation of the blur kernel and the corresponding

image de-convolution procedure for a particular type of blur.

General blur detection and recognition, especially partial blur

detection and analysis, is relatively less explored. Among the

few, Rugna et al. proposed a learning-based method to clas-

sify an input image into blurry and non-blurry regions based

on the observation that the blurry regions are more invariant

to low-pass filtering [4]. Levin et al. explored the gradient

statistics along different directions to segment an input image

into blur/nonblur layers [5]. Dai et al. relied on the result

of image matting to extract the motion blurred region from

the input image [6]. Chakrabarti et al. combined “local fre-

quency” components, local gradient variation and color in-

formation under the MRF segmentation framework to extract

motion blurred regions [7].

The most notable result in the field is Liu et al.’s work

on developing and combining multiple local blur measures

to identify partial image blur [8]. They proposed four local

blur measures, namely Local Power Spectrum Slope (LPSS),

Gradient Histogram Span (GHS), Maximum Saturation (MS)

and Local AutoCorrelation Congruency (LACC) using local

color, gradient and spectral information. The four blur mea-

sures were put into a learning framework with naive Bayes

classifiers for blur/nonblur classification and defocus/motion

blur classification of image patches (LPSS, GHS and MS

were used for blur/nonblur classification and LACC for defo-

cus/motion blur classification). Since Liu’s work, there have

been only limited improvements in addressing the problem,

including the local motion blur measure based on local gradi-

ent energy direction proposed by Chen et al. [9].

Our approach adopts a learning-based framework to ad-

dress the image partial blur detection and classification prob-

lem like Liu et al. did in [8]. The major difference is we have

developed a set of new or enhanced local blur measures with

stronger discriminative power, better across-image stability or

higher computational efficiency. Also, SVM classifiers are

used which are a better choice than the naive Bayes classi-

fiers used by Liu et al.. Experiments show our approach can

achieve higher detection and classification accuracy in much

less time comparing to Liu’s approach.

3. BLUR MEASURES FOR BLUR/NONBLURCLASSIFICATION

3.1. Measure from local saturation

Generally speaking, blurred pixels tend to have less vivid col-

ors than unblurred pixels because of the smoothing effect of

the blurring process. Based on this observation, Liu et al.proposed a per-pixel color saturation metric called Maximum

Saturation (MS) in order to measure the blurriness of an im-

age patch [8]. The MS is defined as the maximum of local

minimums of the R, G and B values being normalized by

their sum, therefor we find it lacks the modeling of the dif-

ference between the R, G and B values. This lack severe-

ly affects its accuracy at measuring the color saturation of

an image dominated by a single color. For example, pixels

of the deep blue sky captured at evening usually have very

low R and G values and mid-range B values, which makes

the resulting MS values extraordinarily high (see Fig. 1). No

mention the max(·) operation in MS mistakenly extends the

(a) Input Image (b) MS (c) b1

Fig. 1. Local color saturation map for blur measurement.

(a) is the input image. (b) is Liu’s MS measure. It has two

problems: 1) abnormal values at pixels dominated by a single

color and 2) the max(·) operation in it extends the saturation

to nearby unsaturated pixels. (c) is the proposed measure b1(eq.(2)). Compared to MS, b1 shows a more natural and ac-

curate modeling of local color saturation.

saturation to nearby unsaturated pixels.

To model local color saturation more elegantly, we pro-

pose to use the following comprehensive color saturation met-

ric defined in the CIELAB color space [10]:

S′p =1

2(tanh(δ · ((Lp − LT ) + (CT − ‖Cp‖2)) + 1) (1)

Here L and C = (a, b)T represent the lightness and the col-

or components in the CIELAB color space respectively. LT

and CT are thresholds for L and C, and δ controls the growth

of the measure. S′p is only effective when correct values of

these parameters are used. After extensive experiments we

find LT = 100, CT = 25 and δ = 1/20 are good settings

for our task. With these settings, our blur measure from lo-

cal saturation is defined as the mean value of S′p in a local

neighborhood P :

b1 =1

NP

∑p∈P

S′p (2)

3.2. Shape of the gradient magnitude distribution

One of the statistical properties of natural images is that their

gradient magnitudes usually follow a heavy-tailed distribu-

tion of wide span [11]. However, blur can change the shape

of this distribution and make it more peaked and narrower by

smoothing out sharp edges [12]. Based on this property, Liu

et al. [8] first fit a two-component Gaussian Mixture Mod-

el (GMM) to the local gradient (magnitude) distribution of

an image patch. After decomposed the GMM using the EM

algorithm [13], they proposed a blur measure called Gradien-

t Histogram Span (GHS) which is defined as the horizontal

span of the larger component, σ1, compensated by local con-

trast (see [8] for details).

We find Liu’s GHS measure has two problems: first, it

assumes that local gradient distribution can be well modeled

by a two-component GMM, but this assumption is not always

valid. For example, the local gradient distribution of a unifor-

m image region (e.g. the blue sky) is actually determined by

(a) Image1 (b) Blurred (c) Unblurred

(d) Image2 (e) Blurred (f) Unblurred

Fig. 2. Gradient magnitude distribution of blurred and un-

blurred regions. (a) and (d) are partially blurred images with

selected blurred and unblurred regions marked by green and

red rectangles respectively. (b) and (e) are gradient distribu-

tion/histogram of the blurred regions in (a) and (d) respective-

ly, and (c) and (f) are those of the unblurred regions. It can be

observed that blurred and unblurred regions are quite differ-

ent at the vertical variation and the curve smoothness of the

gradient magnitude histogram.

the distribution of local image noise which may not be mod-

eled by a GMM properly. Second, dense EM decomposition

at every pixel location is slow and impractical for many ap-

plications.

In view of these shortcomings of GHS, we propose the

following blur measure built upon shape measures of local

gradient distribution directly:

b2 =f1(H)

f2(H) + ε(3)

where H = [h1, · · · , hN ] is the normalized gradient mag-

nitude histogram with N bins of a patch P . f1(H) =Var(H) measures the vertical variation of H , which is a

measure inversely proportional to the horizontal span of H(i.e. width of H) because H is normalized. f2(H) =∑N−1

1 |hi+1−hi|measures the smoothness of the histogram

across bins (i.e. histogram curve smoothness). ε is a s-

mall constant to avoid divide-by-zero errors. Fig. 2 illus-

trates these measures. By comparing Fig. 2(b)/Fig. 2(e) with

Fig. 2(c)/Fig. 2(f), one can observe that histogram vertical

variation f1(H) and curve smoothness f2(H) are pretty dis-

criminative to blurred/unblurred regions and thus it is reason-

able to combine them to build our b2 measure in eq.(3).

Fig. 3 compares the proposed measure b2 (eq.(3)) to Li-

u’s measures including the horizontal span of the larger de-

composed component, σ1, and GHS. It can be seen that the

indication of the blurriness maps built from Liu’s measures

and from ours are similar to each other in general. But close

examination of the σ1 and GHS maps reveal that they are ac-

tually pretty noisy due to the invalidation of the GMM model

assumption they make. Also, each of the σ1 and GHS map-

Fig. 3. Comparison of different blur measures based on gra-

dient magnitude distribution. The top row is for the image

shown in Fig.2(a) (image size:640x480) and bottom row is

for the image in Fig.2(d) (image size: 640x430). In each row,

from left to right are the blurriness maps built from Liu’s mea-

sures σ1 and GHS, and from our measure b2 (eq.(3)). (Maps

are displayed in log space for better visualization.)

s shown here took about 1.5 hours to compute, while the b2maps took only < 200 seconds, demonstrating the proposed

measure is much more efficient.

3.3. The power spectrum slope

The last measure we use to identify blur is the power spec-

trum slope which quantifies the decay of signals across fre-

quencies in the Fourier domain. We argue that this measure

has a stable range across a wide variety of images, which dis-

putes the claim made by Liu et al. against this measure in

[8]. Given an image I of size MxN , its squared magnitude

in the frequency domain is: S(u, v) = 1M ·N I(u, v) , where

I(u, v) = F(I) is the Discrete Fourier transform (DFT) of

I . Letting S(f, θ) be the corresponding polar representation

of S(u, v) with u = f cos θ and v = f sin θ, the power spec-

trum S is then the summation of S(f, θ) over all directions

θ [14]: S(f) =∑

θ S(f, θ) � κf−α , where κ is an ampli-

tude scaling factor for each orientation and α is the frequency

exponent. Empirical data show that for natural images the

power spectrum S(f) falls off at 1/f2 with increasing f [14].

This corresponds to a curve with slope of≈ −2 in the log-log

coordinates, that is, α ≈ 2.

A blurred image usually has a larger α because of the loss

of high frequency signals. Discriminative studies showed that

even for small image patches this relationship between blur-

riness and α holds [15], which encouraged Liu et al. to ex-

plore this relationship for patch-based partial blur detection

in [8]. However, Liu et al. claimed that direct usage of α is

inappropriate because α varies by image content. They thus

suggested a relative measure of blurriness called Local Power

Spectrum Slope (LPSS) that normalizes each local αP for a

patch P by the global αI of the image I .

Surprisingly, we have reached a different conclusion to

Liu’s about αP and LPSS after testing both of them on our

dataset. Our experiments show the power spectrum slope αP

Fig. 4. Four partially blurred images and their power spec-

trum slope (αP ) and LPSS maps. For each row, left is the

image, center is the αP map, and right is the LPSS map. As

shown by colorbars on the right of the maps the scale of the

αP map is relatively stable for different images, while the

scale of the LPSS map is much less stable and varies dramat-

ically across the images.

is quite stable across a wide variety of images, while Liu’s

relative measure LSPP is on the contrary quite unstable. Fig. 4

shows our findings. Thus our third local blur measure is

b3 = αP . (4)

4. BLUR MEASURES FOR MOTION/DEFOCUSBLUR CLASSIFICATION

A discriminative feature of motion blur is that the correspond-

ing local gradient orientation distribution is more peaky than

normal, with the peak at the direction perpendicular to the

motion direction. Chen et al. only used the gradient ener-

gy along the estimated local motion direction for motion blur

detection [9]. We think this energy only partially reflects the

above feature of motion blur, and the ratio between the ener-

gies at the directions along and perpendicular to the motion

direction should also be used to fully reflect the feature. The

image gradient at a point (x, y) at a direction θ is represent-

ed as: ΔI(x, y) · θ = [Ix(x, y)Iy(x, y)][cos θ sin θ]T . Then,

within a local neighborhood P one can build a directional en-

ergy measurement of squared directional gradients [9]:

E(P ) =∑P

([Ix(x, y) Iy(x, y)][cos θ sin θ]T )2

=[cos(θ) sin(θ)

]D(x, y)

[cos(θ)sin(θ)

](5)

where

D =

[d11 d12d21 d22

]=∑P

[I2x IxIyIxIy I2y

](6)

is the local autocorrelation matrix. E(P ) can further be rep-

resented as a function of θ:

f(θ) = E(P ) =1

2(d12 + d21) sin(2θ) + (d22 − d11) sin

2(θ)

(7)

Its minimum value can be obtained by setting∂f(θ)∂θ =

0, which yields θ̂ = θbase + nπ2 , where θbase =

12 tan

−1(

d12+d21

d11−d22

). In [9], the local motion (blur) direction

is estimated as:

θ̂motion =

{θbase if f(θbase) ≤ f(θbase +

π2 )

θbase +π2 if f(θbase) > f(θbase +

π2 )

and the corresponding energy f(θ̂motion) is used to measure

motion blur.

By contrast, we propose to use both the energy along the

estimated motion direction and the ratio of energies at ex-

treme directions for blur measurement. We define:

m1 = log(f(θ̂motion)) (8)

and

m2 = log

(f(θ̂motion + π

2 )

f(θ̂motion) + ε

)(9)

where log(·) is used to bring the measures to an easier-to-

manage range. One advantage of the proposed m2 measure

is that it is a relative measure which is not affected by the

absolute intensity of a patch.

In addition to the above two measures, we observe that

Liu’s local autocorrelation congruency (LACC) measure [8]

can encode the gradient directional distribution of a larger

neighborhood. And we directly use it as our third measure

for motion blur:

m3 = log(LACC(P )) (10)

where LACC is defined as: LACC = V ar(hist(θ)) , with

hist(θ) being a directional response histogram for patch Pwhich is constructed from the eigen-decomposition of the lo-

cal autocorrelation matrix D (Eq.(6)).

5. EXPERIMENTAL RESULTS

As in Liu et al.’s work [8], we realize image partial blur i-

dentification via patch-based classification. We define three

blur labels for classification: sharp, motion blur or defocus

blur. Given an input image patch, we perform blur/nonblur

classification on it first. If a patch is identified as blurred, we

then perform motion/defocus blur classification on it to iden-

tify the blur type. We use a linear SVM classifier with the blur

(a) Blur/nonblur classification (b) Motion/defocus classification

Fig. 5. ROC curves for blur/nonblur and motion/defocus clas-

sification.

measures b1, b2 and b3 for blur/nonblur classification, and use

another linear SVM classifier with the motion blur measures

m1, m2 and m3 for motion/defocus blur classification.

The dataset used in our experiments was created from 200

partially blurred images downloaded from www.flickr.com. Images are rescaled so that their maximum dimension

is 640 pixels. We develop a graphic tool for convenient s-

election and labeling of the sharp/blurred segments in these

images. The user selects a region and indicates whether it

is sharp, motion blurred or defocus blurred by drawing in-

dicating polygons of different colors. For selecting large re-

gions or regions of irregular shape, the tool allows the user to

finish the drawing in several steps and automatically merges

overlapping input polygons of the same color. The tool al-

so supports converting a finer pixel-level segment map to a

coarser patch-label segment map by analyzing the composi-

tion of pixel labels within each block. We will make this tool

available for download by other researchers.

The labelling tool automatically generates patch data from

the image dataset to form the traing/test datasets. Patch size

and patch sampling interval are both controllable parameters

and are set to 51x51 pixels in all our experiments. In total,

we obtain a dataset composed of 1992 sharp patches, 1019

motion blur patches and 2477 defocus blur patches. We ran-

domly selected 70% of the patch data for training and the re-

maining 30% for testing. We compared our approach to Li-

u’s approach whose hyper-parameters are also auto-selected

by 5-fold cross validation on the same training dataset. We

compute the ROC curves (true positive rate (TPR) v.s. false

positive rate (FPR)) (Fig. 5) and the accuracy rate (AR) [8] on

the test dataset to measure the performance of both approach-

es. Our approach has achieved 84.6% and 80.2% accuracy

rates for blur/nonblur and motion/defocus blur classifications

on the test dataset. In comparison, Liu’s approach can only

achieve 80.4% and 28.3% accuracy rates respectively. Note

that our approach achieves this performance gain over Liu’s

approach with much lower computational cost (see Sec. 3.2).

Example results for image partial blur segmentation using

the learned classified are shown in Fig. 6. Although the ROC

curves show that Liu’s approach and our approach are compa-

rable at patch-level blur/nonblur classifying, we can tell from

these examples that the sharp regions (marked in blue) iden-

tified by Liu’s approach are less tight around the true object

than ours. This is shown by the flower in Fig.6(c) and by the

body of the bee in Fig.6(i). Liu’s approach also mis-classifies

many sharp and defocus blur region to motion blur (marked

in red). This was confirmed by the very low (28.3%) mo-

tion/defocus blur classification accuracy rate, and is demon-

strated here again by the segmentation examples. For exam-

ple, in Figs.6(b), 6(d) and 6(f), much of the tree areas, which

are pretty clear and sharp, are mis-classified as motion blur

by Liu’s approach. Also, Liu’s classification of the defocus

blurred areas in Figs.6(a), 6(c) and 6(i) are completely wrong.

We think this is because Liu’s approach uses only one motion

blur feature and a naive Bayes classifier for motion/defocus

blur classification, both of which are not very reliable and

prone to errors. In contrast, our approach uses three motion

blur features and SVM-based classification and thus demon-

strates more robust performance.

Figs.6(g) and 6(h) are complicated examples because they

contain all three kinds of regions: sharp, motion blurred

and defocus blurred. Our approach works very well for

these complicated cases, while Liu’s approach always mis-

classifies the defocus blurred regions as motion blur. For

three other examples in Figs.6(b), 6(e) and 6(f), our approach

produces competitive results to those of Liu’s approach. For

the mis-classified regions of these examples Liu’s approach

prefers to classify them as motion blurred while ours prefers

defocus blurred.

6. CONCLUSION

In this paper, we revisited the image partial blur detection and

classification problem and proposed a set of new or enhanced

local blur measures based on our analysis of the relation-

ship between image blur and various image properties such

as saturation, gradient and power spectrum. We embedded

these blur measures into an SVM-based learning approach for

classifying a given image patch as sharp, defocus blurred or

motion blurred. We developed an easy-to-use graphical seg-

mentation tool and created large-size training/test datasets for

patch-based classification using the tool. Experiments on the

datasets show our approach achieves a significant improve-

ment over the state-of-the-art approach of Liu et al. [8]. Our

approach was also applied to the problem of segmenting par-

tially blurred images and correctly labeling blur types. Real

and promising segmentation results were analyzed and out-

performed Liu’s approach.

7. REFERENCES

[1] Suk Hwan Lim, Jonathan Yen, and Peng Wu, “Detection

of out-of-focus digital photographs,” Tech. Rep. HPL-

2005-14, HP Laboratories Palo Alto, 2005.

(a) example1 (b) exam-

ple2

(c) example3

(d) example4 (e) example5 (f) exam-

ple6

(g) example7 (h) example8 (i) example9

Fig. 6. Image partial blur segmentation using the trained classifiers. For each example, from left to right are: input image,

segmentation output of Liu’s approach and segmentation output of our approach. Unblurred regions are marked in blue, motion

blurred region in red, and defocus blurred regions in yellow.

[2] Rony Ferzli and Lina J. Karam, “A no-reference objec-

tive image sharpness metric based on the notion of just

noticeable blur (jnb),” IEEE Transaction on Image Pro-cessing, vol. 18, no. 4, pp. 717–728, 2009.

[3] Pina Marziliano Frederic, Frederic Dufaux, Stefan Win-

kler, Touradj Ebrahimi, and Genimedia Sa, “A no-

reference perceptual blur metric,” in Proc. 2002 IEEEInternational Conference on Image Processing, 2002, p-

p. 57–60.

[4] Jerome Da Rugna and Hubert Konik, “Automatic blur

detection for meta-data extraction in content-based re-

trieval context,” in Proc. SPIE, 2003, vol. 5304, pp.

285–294.

[5] Anat Levin, “Blind motion deblurring using image s-

tatistics,” in 2006 Advances in Neural Information Pro-cessing Systems, 2006, pp. 841–848.

[6] Shengyang Dai and Ying Wu, “Estimating space-variant

motion blur without deblurring,” in Proc. 15th IEEE Int.Conf. on Image Processing, 2008, pp. 661–664.

[7] Ayan Chakrabarti, Todd Zickler, and William T. Free-

man, “Correcting over-exposure in photographs,” in

Proc. 2010 IEEE Conference on Computer Vision andPattern Recognition, 2010, pp. 2512–2519.

[8] Renting Liu, Zhaorong Li, and Jiaya Jia, “Image partial

blur detection and classification,” in Proc. 2008 IEEEConference on Computer Vision and Pattern Recogni-tion, 2008, pp. 1–8.

[9] Xiaogang Chen, Jie Yang, Qiang Wu, and Jiajia Zhao,

“Motion blur detection based on lowest directional high-

frequency energy,” in Proc. 17th IEEE Int. Conf. onImage Processing, 2010, pp. 2533 – 2536.

[10] Dong Guo, Yuan Cheng, Shaojie Zhuo, and Terence

Sim, “Correcting over-exposure in photographs,” in

Proc. 2010 IEEE Conference on Computer Vision andPattern Recognition, 2010, pp. 515–521.

[11] Stefan Roth and Michael J. Black, “Fields of expert-

s: a framework for learning image priors,” in Proc.2005 IEEE Conference on Computer Vision and PatternRecognition, 2005, vol. 2, pp. 860–867.

[12] R. Fergus, B. Singh, A. Hertzmann, S. Roweis, and

W. Freeman, “Removing camera shake from a single

photograph,” ACM Transaction on Graphics, vol. 25,

no. 3, pp. 787–794, 2006.

[13] J. A. Bilmes, “A gentle tutorial of the EM algorithm

and its application to parameter estimation for gaussian

mixture and hidden markov models,” Tech. Rep. ICSI-

TR-97-021, University of California at Berkeley, 1997.

[14] David J. Field, “Relations between the statistics of natu-

ral images and the response properties of cortical cells,”

Journal of Optical Society of America, vol. 4, no. 12, pp.

2379–2394, 1987.

[15] B. Hansen and R. Hess, “Discrimination of amplitude

spectrum slope in the fovea and parafovea and the local

amplitude distributions of natural scene imagery,” Jour-nal of Vision, vol. 6, no. 7, pp. 696–711, 2006.