An Edge-preserving Filtering Framework for Visibility Restoration

Linchao Bao1, Yibing Song1, Qingxiong Yang1, Narendra Ahuja2Dept. of Computer Science, City University of Hong Kong1

Dept. of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign2

linchaobao@gmail.com, yibisong, qiyang@cityu.edu.hk, ahuja@vision.ai.uiuc.edu

Abstract

In this paper, we propose a new framework for edge-preserving texture-smoothing filtering to improve thevisibility of images in presence of haze or fog. The pro-posed framework can effectively achieve strong texturesmoothing while keeping edges sharp, and any low-passfilter can be directly integrated into the framework. Ourexperiments with three popular low-pass filters (Gaus-sian filter, median filter and bilateral filter) show thatour framework can better preserve the contrast aroundedges than the original filters. The visibility restorationalgorithm based on our framework outperforms exist-ing filtering-based algorithm without losing the speedadvantage compared to non-filtering-based algorithms.

1. IntroductionThe presence of haze or fog in images can degrade

the visibility of photographs and may hamper theperformances of some image processing algorithmsin computer vision applications. It is important andchallenging to develop algorithms that can improvethe visibility of haze images. Existing visibilityrestoration algorithms using multiple images [5] orscene depth information [3] are rather limited becauseof the acquisition of additional information. Recently,several algorithms have been proposed to solve theproblem using a single haze image by introducingsome assumptions or priors. Tan [7] turns the probleminto an optimization problem by assuming that thevisibility improved image should have better contrastand the airlight varies smoothly depending on theobject depth in the scene. Fattal [1] solves the problembased on the assumption that object surface shadingand medium transmission function of haze should belocally uncorrelated. He et al. [2] introduce a darkchannel prior which comes from their observationthat the minimum color component over eroded colorchannels of outdoor haze image turns out to be closelyrelated to the medium transmission map of haze. Theyuse the dark channel to estimate a rough medium

transmission map and then refine it using soft mattingalgorithms. However, the common disadvantage ofthese three algorithms are that they are too slow tofulfil the demands of real-time applications. Basedon these pioneering works, Tarel et al. [8] propose afast algorithm which treats the problem as a filteringproblem and thus can achieve much faster speed. Thealgorithm estimates the medium transmission mapbased on edge-preserving smoothing, and then restoresthe image using the medium transmission map. Thecore step is the edge-preserving smoothing step, wherethey utilize median filtering.

(a) Haze image (b) Tarel’s result [8] (c) Our result

(d) Tarel’s result [8] (zoom in) (e) Our result (zoom in)

Figure 1. Visibility restoration results. Appar-ently, our method outperforms Tarel’s algorithm[8] around large depth jumps.

The objective of using edge-preserving filteringin the visibility restoration algorithm is to smoothtextured regions while preserving large scene depthdiscontinuities. Two popular edge-preserving filteringmethods are median filtering and bilateral filtering.Unfortunately, both filtering methods cannot workwell in visibility restoration. Median filter has strongsmoothing ability but can blur edges, while bilateralfilter with small range variance cannot achieve enoughsmoothing on textured regions in the visibility restora-tion algorithm and that with large range variance willalso blur edges. Figure 1 (b) shows the restorationresults of Tarel’s algorithm [8] that utilizes medianfiltering. Note that their algorithm cannot remove hazenear large scene depth discontinuities because medianfiltering cannot preserve edges well. In this paper,we present a new edge-preserving texture-smoothingfiltering framework and show that it can generate betterresults in visibility restoration of haze images thanexisting methods.

2. Background2.1. Edge-preserving Filtering

The main idea of the median filtering is to runthrough all pixels, replacing each pixel value with themedian of neighboring pixel values. Bilateral filtering[9] is done by replacing the intensity value of a pixelby the average of the values of other pixels weighted bytheir spatial distance and intensity similarity to the orig-inal pixel. Specifically, the bilateral filter is defined anormalized convolution in which the weighting for eachpixel q is determined by the spatial distance from thecenter pixel p, as well as its relative difference in inten-sity. Let Ip be the intensity at pixel p and IFp be thefiltered value,

IFp =

∑q∈Ω

GσS′ (∥p− q∥)GσR(|Ip − Iq|)Iq∑

q∈Ω

GσS′ (∥p− q∥)GσR(|Ip − Iq|)

, (1)

where the spatial and range weighting functions GσS′

and GσRare often Gaussian, and σR and σS′ are

the range and spatial variances, respectively. In thispaper, the image intensities are normalized such thatIp ∈ [0, 1] and σR is normally chosen between [0, 1].Let the image width and height be w and h, we chooseσS′ = σS · min(h,w), such that σS is also normallychosen between [0, 1]. Besides bilateral filter, we alsouse σS′ to represent the spatial variance of Gaussianfilter and the radius of median filter in this paper.

2.2. Visibility Restoration ModelThe widely adopted model for the formation of im-

age in presence of haze or fog is as follows [4]:

Ip = Jptp +A(1− tp), (2)

where I is the haze image, J is the scene radiance (theactual appearance of the scene without haze), A isthe atmospheric light (commonly assumed as a globalconstant), tp is the medium transmission functiondepending on the scene depth. When the image isrepresented as a RGB three channels color image, Ip,Jp, and A are vectors and tp is a scalar. The physicalprocess of this equation can be described as follows:the actual radiance Jp at a point p in the scene, isattenuated by the atmosphere when passing throughthe haze, and shifted by an effect of atmospheric lightA which is accumulated when the radiance passingthrough the haze. The first term Jptp describes thedirect attenuation of scene radiance, while the secondterm A(1− tp) describes the effect of atmospheric lightand is commonly called airlight. The goal of visibilityrestoration or dehazing algorithms is to estimate Jgiven a single haze image I. In order to estimate J, thecommon way is to first estimate tp and A.

As can be seen from equation (2), the problem of es-timating J using only I is severely ill-posed. Additionalassumptions or priors need to be introduced to solve it.As mentioned in section 1, several methods have beenproposed to solve it by introducing different assump-tions or priors. In this paper, we follow the filtering-based method [8] and the additional assumptions andpriors are hereby described explicitly as follows:• The dark channel [2] of the haze image I is a rough

estimation of term (1− tp) in equation (2);• The depth of scene objects in an image varies

smoothly except over large depth jumps [7];• All RGB color images have been white balanced,

which implies that the atmospheric light A can besimply set to (1, 1, 1) [8].

3. Our Filtering FrameworkIn this section, we present the details of our

edge-preserving filtering framework and show howit works differently from the original filters. Givena grayscale image I as shown in Figure 3 (a) (if theinput is multi-channel image, we process each channelindependently), we first detect the range of the imageintensity values, say [Imin, Imax]. We next sweep afamily of planes at different image intensity levelsIk ∈ [Imin, Imax], k = 0, 1, ..., N − 1 across theimage intensity range space. The distance betweenneighboring planes is set to Imax−Imin

N−1 , where N is aconstant. Figure 2 (b) shows such an example whenN = 5. Smaller N results in larger quantization errorwhile larger N increases the running time. WhenN → +∞ or N = 256 for 8-bit grayscale images, thequantization error is zero. Also, stronger smoothingusually requires smaller N .

We define the distance function of a voxel[I(x, y), x, y] on a plane with intensity level Ik as the

(a) 3D view of Figure 3 (a) (b) Established planes (N = 5)

(c) Distance maps D(Ik) (d) Compute filtered image

Figure 2. Our filtering framework.

truncated Euclidean distance between the voxel and theplane (as shown in Figure 2 (c)):

D(Ik, x, y) = min(|Ik − I(x, y)|, η · |Imax − Imin|),(3)

where η ∈ (0, 1] is a constant used to reject outliersand when η = 1, the distance function is equal to theirEuclidean distance.

At each intensity plane, we obtain a 2D distance mapD(Ik). A low pass filter F is then used to smoothen thisdistance map to suppress noise:

DF (Ik) = F(D(Ik)). (4)

Then, the plane with minimum distance value at eachpixel location [x, y] is located by calculating

K(x, y) = argmink=0,1,...,N−1

DF (Ik, x, y). (5)

Let intensity levels i0 = IK(x,y), i+ = IK(x,y)+1, i− =IK(x,y)−1. Assume that the smoothed distance functionDF (Ik, x, y) at each pixel location [x, y] is quadraticpolynomial with respect to the intensity value Ik, theintensity value corresponding to the minimum of thedistance function can be approximated using quadraticpolynomial interpolation (curve fitting):

IF (x, y) = i0−DF (i+, x, y)−DF (i−, x, y)

2(DF (i+, x, y) +DF (i−, x, y)− 2DF (i0, x, y)),

(6)

which is the final filtered result of our framework ateach pixel location [x, y].

(a) Original (b) Gaussian (c) Median (d) Bilateral

(e) Original (f) Our Gaussian (g) Our median (h) Our bilateral

Figure 3. Filtered results of a synthetic noisyimage. (a) is the original grayscale image, andis visualized using a color map in (e) for clarity.(b)-(d): the filtered image obtained using Gaus-sian filter, median filter, and bilateral filter, re-spectively. (f)-(h): filtered images obtained usingour edge-preserving filtering framework with thecorresponding filters in (b)-(d).

Apparently, any filter F can be integrated into thisframework by using it to smoothen the distance mapD(Ik) as shown in equation (4). Figure 3 presents theexperimental filtered results of our framework for asynthetic noisy image. In this experiment, the numberof intensity planes N is set to 16 and η = 0.1. Theparameter settings for low-pass filters are σS = 0.05and as for bilateral filter, σR = 0.2. As can be seen inFigure 3, the use of our framework maintains the sharpintensity edges and greatly suppresses the noise insideeach region.

4. Filtering-based Visibility RestorationIn Tarel’s algorithm [8], they first calculate the min-

imum color components W of the input image I by

Wp = minc∈r,g,b

(Icp) (7)

where p is the location of each pixel in the image. Thenthey perform the following filtering on W to get B:

A = median(W ), (8)

B = A−median(|W −A|). (9)

B is used to estimate the transmission map tp by

tp = 1−min(ϵ ·Bp,Wp), (10)

where the factor ϵ in [0, 1] is to control the strength ofthe restoration. Finally, each channel Jc of the restoredimage J can be calculated by

Jcp = 1− (1− Icp)/tp, (11)

where c ∈ r, g, b for RGB images. A followedpost-processing step of tone mapping is suggested [8]

to make the color look similar to the input image.

Due to the twice median filtering (equation (8) and(9)) employed in Tarel’s algorithm, it cannot removehaze near large depth jumps in the scene. On the otherhand, bilateral filter can better preserve object bound-aries when a small σR is used [9], but most of the tex-tures are also kept thus it cannot provide enough tex-ture smoothing in the visibility restoration algorithm.More aggressive smoothing can be achieved by increas-ing σR, but the increasing σR reduces the ability of thebilateral filter to preserve edges and some of the edgesbecome blurred, as evident from Figure 3 (d). By em-ploying our filtering framework, we modify the process-ing steps of visibility restoration as follows:1) Calculate W using equation (7);2) Get B by filtering W using our framework;3) Estimate t using equation (10);4) Calculate J using equation (11).

5. Experimental ResultsIn all the experiments conducted in this paper, we

use bilateral filter as the operator F in our filteringframework. We set the number of intensity planes Nto 16, and set the parameter η in equation (3) to 0.1.The parameter ϵ in equation (10) is set to 0.95. Figure1 (b) and (c) show the results of restoring an imageusing Tarel’s algorithm and our algorithm, respectively.The filtering parameters for both algorithms are set toσS = 0.05, σR = 0.1 (σR is only for our algorithm).Compared with Tarel’s algorithm, it is obvious thatour algorithm can more effectively remove haze nearlarge depth discontinuities. Figure 4 shows the resultcomparison of another image.

The speed of our algorithm is comparable to Tarel’salgorithm, while other non-filtering-based visibilityrestoration algorithms mentioned in Section 1 are notcomparable to our filtering-based algorithms [8]. Weimplement both our algorithm and Tarel’s algorithmin C++. The bilateral filter in our algorithm is imple-mented using an O(1) complexity per pixel algorithm[10] and the median filter in Tarel’s algorithm isimplemented by a fast O(1) median algorithm [6]. Ex-periments show that our algorithm is only about 2.2×slower than Tarel’s algorithm on average. Specifically,the average running time for the image shown in Figure4 (1024 × 768 pixels) are 0.78 seconds (ours) and0.35 seconds (Tarel’s) on a PC with Intel Core i7-26003.4GHz CPU and 4GB RAM, respectively.

6. ConclusionWe have described a filtering framework for edge-

preserving smoothing, and demonstrated the benefitsof using this framework when applying it to visibilityrestoration algorithm, where strong texture smoothing

(a) Haze image (b) Tarel’s result [8] (c) Our result

(d) Tarel’s result [8] (zoom in) (e) Our result (zoom in)

Figure 4. Visibility restoration results. Note thatthere is clear improvement around the leaves.

is required and the contrast around edges should bepreserved. In the future, we intend to further improveour framework and apply it to other applications.

References[1] R. Fattal. Single image dehazing. In ACM Transactions

on Graphics (TOG), volume 27, page 72. ACM, 2008.[2] K. He, J. Sun, and X. Tang. Single image haze removal

using dark channel prior. In CVPR, pages 1956–1963.IEEE, 2009.

[3] J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O. Deussen, M. Uyttendaele, and D. Lischinski.Deep photo: Model-based photograph enhancementand viewing. In ACM Transactions on Graphics (TOG),volume 27, page 116. ACM, 2008.

[4] S. Narasimhan and S. Nayar. Vision and the atmo-sphere. International Journal of Computer Vision,48(3):233–254, 2002.

[5] S. Narasimhan and S. Nayar. Contrast restoration ofweather degraded images. IEEE Transactions on Pat-tern Analysis and Machine Intelligence, 25(6):713–724,2003.

[6] S. Perreault and P. Hebert. Median filtering in con-stant time. IEEE Transactions on Image Processing,16(9):2389–2394, 2007.

[7] R. Tan. Visibility in bad weather from a single image.In CVPR, pages 1–8. IEEE, 2008.

[8] J. Tarel and N. Hautiere. Fast visibility restoration froma single color or gray level image. In ICCV, pages 2201–2208. IEEE, 2009.

[9] C. Tomasi and R. Manduchi. Bilateral filtering for grayand color images. In ICCV, pages 839–846. IEEE,1998.

[10] Q. Yang, K. Tan, and N. Ahuja. Real-time o (1) bilateralfiltering. In CVPR, pages 557–564. IEEE, 2009.