Separating linear features from textured backgrounds

Journal of Electronic Imaging 15(2), 023012 (Apr–Jun 2006)

Downloaded

Separating linear features from textured backgrounds

Bhavani PidaparthiRaghuveer M. Rao

Honghong PengRochester Institute of TechnologyElectrical Engineering Department

Rochester, New York 14623

Abstract. The problem of separating linear features from a texturedbackground is of importance in many applications. It has beenshown that the Fourier transform can be used in conjunction withpolar transformation to “lift” linear features from the background tex-ture. However, while the Fourier transform works well with lines thatare spread throughout the entire image, it is less effective when thelinear features are of varied length and thickness. We propose ap-proaches based on a windowed Fourier transform and waveletpacket decomposition to lift randomly located lines of varied lengthsand thickness. The reasoning underlying the development of theapproaches is presented along with comparative examples. © 2006SPIE and IS&T. �DOI: 10.1117/1.2204073�

1 Introduction

The processing of linear features in textured images hasbeen addressed using various approaches in theliterature.1–4 These approaches are useful in detecting, ex-tracting, and separating linear features from texturedbackgrounds.5 Campos and Kasparis2 proposed a techniqueof line detection in textures that uses the Hough transform.Chien and Li3 proposed a wavelet-based approach to linedetection in gray-scale images. Their technique exploits theproperty of the wavelet transform to detect lines and edgesin the detail coefficients. Hou and Bamberger4 investigateda filter-based approach to linear feature extraction. How-ever, while these techniques are aimed at detecting linearfeatures in images, Beyerer and León1 proposed an elegantimage processing technique, which we refer to as the BLapproach, to lift linear features from a texture to give twoimages; one containing the linear features and the othercontaining the textured background. Separating linear fea-tures from textured backgrounds also offers potential appli-cations in scene animation,6 lithography, layout design, andpattern recognition.

The Fourier-transform-based BL approach assumes thatthe randomly located linear features run through the entireimage. As we demonstrate, this results in it being less ef-fective in separating linear features of varied lengths andthickness. This paper presents two approaches—one basedon a windowed Fourier transform and the other based onwavelet packet decomposition—to separate linear features

Paper 03145 received Oct. 21, 2003; revised manuscript received Oct. 13,2005; accepted for publication Nov. 15, 2005; published online Jun. 9,2006.
1017-9909/2006/15�2�/023012/7/$22.00 © 2006 SPIE and IS&T.
Journal of Electronic Imaging 023012-

From: http://electronicimaging.spiedigitallibrary.org/ on 04/15/2013 Terms of

of varied lengths and thickness effectively. Mathematicalanalyses as well as experimental results are provided insupport of this claim.

The BL approach is presented in the next section. Thelimitations of the technique are presented in Sec. 3. Theproposed Fourier transform block approach and thewavelet-based algorithm are presented in Sec. 4. Resultsare presented in Sec. 5 and concluding remarks in Sec. 6.

2 Overview of the Fourier Transform Techniqueof Line Separation in Textures

2.1 Signal ModelConsider an image consisting of straight lines in a texturedbackground such as the one shown in Fig. 1. Notice that thelines run unbroken throughout the image. Suppose we areinterested in separating the image in Fig. 1 into the line andbackground textures. Following Beyerer and León,1 the im-age is mathematically represented as

g�x� = ��xT − d� + b�x� , �1�

where x= �x1x2�T, g�x� is the texture with linear features,b�x� is the background texture, and for each line i, we have�omitting the subscript i�:

1. The cross profile of the line is ��.�.2. The unit normal vector along the perpendicular

from the origin to the straight line is n. It is ex-

Fig. 1 Texture with lines extending throughout the image.

Apr–Jun 2006/Vol. 15(2)1

Use: http://spiedl.org/terms

Pidaparthi, Rao, and Peng: Separating linear features from textured backgrounds

Downloaded

pressed as n= �cos � sin ��T, where � is the anglebetween the normal vector and the x axis.

3. The perpendicular distance between the line andthe origin is d.

The isotropy of b�x� means that the magnitude of the cor-responding Fourier transform is symmetric with respect torotation as follows:

B�f� = I�b�x�� = B��f�� , �2�

where �f� denotes the Euclidean norm of the spatial fre-quency vector F= �f1 , f2�T and ��.� is the radial profile ofB�f�.

For simplicity we will consider a single line. The Fou-rier transform of g�x� is given by

I�g�x�� = G�f� = ��fTn��fTp� � exp �− j2�d fTn� + B��f�� ,

�3�

where

�� =� �� exp �− j2�� d � � R ,

and p= �−sin � cos ��T is the unit vector along the line.Since the � function dominates G�f� for fTp=0 and is zeroelsewhere, the magnitude of G�f� in Eq. �3� is*

G�f� = ��f1/cos ��cos �

��fTp� + ��f�� . �4�

The derivation of the proceeding is based on spatial scalingproperties of the 2-D Fourier transform.7 The first term isconcentrated along the straight line fTp=0, which passesthrough the origin. The removal of the phase eliminates thedependence on the distance d. The background b�x� istransformed into a rotationally symmetrical componentcentered on the origin.

Suppose we now transform to polar coordinates as

�f1, f2� = � cos �, sin �� , �5�

where �� 0,�� is the angle of the spatial frequency vectorand �R is the signed, radial frequency coordinate. Wehave

��fTp� = ��sin �� − �� ,

G� cos �, sin �� = �� cos �/cos ��cos �

�� sin �� − ��

+ �� . �6�

Also,

�� sin �� − �� = �n

�� − �� + n��/ . �7�

From Eqs. �6� and �7�,

* T
It is also equal to ��f2 / sin �� / sin � ��f p�+��f � �.


I,��G� cos �, sin ��

= Ip��1

��

k=−�

�

�� f� −k

�

�exp �− j2�f��

+ Ip�B��f�� , �8�

where I,� is the 2-D Fourier transform with respect to the-� coordinates, and fp and f�, respectively, are the corre-sponding frequency variables. The sum of an infinite num-ber of impulses in Eq. �8� arises from the fact that theFourier transform of a train of impulses is also a train ofimpulses, and we do have such a train in Eq. �7�. Hence,linear features are mapped to vertical lines in the f-f� do-main. At the same time, the background features are mainlymapped into a vertical line impulse along the f axis. Notethat this observation is independent of

1. The slope and position of the linear feature in theoriginal image.

2. The number of distinct linear features. Multiplelines merely result in additional contributions tothe first term in Eq. �8�. These additional contribu-tions will also be vertically oriented as this term.

The more prominent a linear feature is, that is, the closer��.� is to an impulse in Eq. �1�, the closer are the corre-sponding vertical lines in the f-f� domain to impulsesthemselves. This is because with ��.� close to an impulse,its Fourier transform is spread out but the second Fouriertransformation in Eq. �8� yields something close to an im-pulse. Thus, linear features can be extracted by retainingthe vertical lines and suppressing the value along the f

axis.

2.2 AlgorithmA Fourier transform is first performed on the input image.Then, a rectangular to polar coordinate transform is per-formed. To keep the algorithm independent of the knowl-edge of ��, its dynamic range is attenuated by applyinga logarithm. A value of 1 is added prior to taking the loga-rithm to avoid the singularity of the logarithm at zero. AFourier transform is done on the image, such that all linesare transformed to delta functions along the x axis. Thetransformed image is then mask filtered to suppress thedelta functions. The filtered image is then passed through asequence of transformations, which are the inverse Fouriertransform, an exponential transform, and a coordinatetransform. The resultant spectrum, B��f� is an estimate ofthe background spectrum and is used to separate the spec-trum of the input image into line and background spec-trums.

The ratio G�f� / B��f� is compared to a threshold foreach discrete frequency. If the ratio exceeds , the corre-sponding spectral component is assigned to the line pattern,otherwise to the background texture.

L��f� = �G�f� if �G�f�/B��f� �

0 otherwise.� �9�

Also,

Apr–Jun 2006/Vol. 15(2)2



Downloaded

B��f� = �G�f� if �G�f�/B��f� �


It has been demonstrated that for =�2, good separation ofthe line and background textures is observed.

3 Limitations of the Fourier Transform ApproachThe Fourier transform technique of separating linear fea-tures works effectively for randomly located lines that ex-tend across the entire image. However, when the linear fea-tures are of varied length and thickness, this approach isless effective in bringing about a good separation of the lineand background textures. This is a potential limitation ofthe Fourier transform approach in applications since tex-tures of wood and other surfaces have linear features,which are of random lengths on the surface and are also ofvaried thickness. A mathematical explanation of this claimis provided in the following.

Unlike the development in Beyerer and León,1 wherethe signal model is in continuous space, we develop thesignal model in a bounded discrete space consistent withactual image representation. Consider a straight lineg�n1 ,n2� defined as

g�n1,n2� = ��n1 − an2� �k1=0

M−1

�k2=0

M−1

��n1 − k1,n2 − k2� , �11�

where a is an integer. The function g�n1 ,n2� is mostly zeroexcept for a straight line contained in a M �M window.

Taking the Fourier transform on both sides,

G��1,�2� = �n1=−�

�

�n2=−�

�

g�n1,n2� exp �− j��1n1 + �2n2��

= �n2=0

M−1

exp �− j�2n2� �n1=0

M−1

��n1 − an2�

�exp �− j�1n1� . �12�

Fig. 2 Short line in an image.

We have



�n1=0

M−1

��n1 − an2� exp �− j�1n1� = exp �− j�1an2� for n1

= �an2 and 0 � n2 � M − 1


From Eqs. �12� and �13�,

G��1,�2� = �n2=0

M−1

exp �− jn2��2 + a�1�� . �14�

Therefore,

G��1,�2� = A exp �−j��2 + a�2�M

2 sin ��2 + a�1�M/2�

sin ��2 + a�1/2�.

�15�

If g�n1 ,n2� is truncated to a square region of size N�N, thediscrete Fourier transform of g�n1 ,n2� is given by

�G�k1,k2� = G��1,�2��1=2�k1

N

�2=2�k2

N

,

�G�k1,k2� = A exp �− j��k2 + ak1��

�sin ��k2 + ak1�M/N�sin ��k2 + ak1�/N�

M � N . �16�

Equation �16� shows that the N-point discrete Fourier trans-form of g�n1 ,n2� when the straight line is of length M,where M �N, is a 2-D sine over sine �somewhat similar toa sinc� function that is spread out in the frequency domain.If M =N, the preceding becomes

G�k1,k2� = A��k2 + ak1� . �17�

Equation �17� shows that the N-point discrete Fourier trans-form �DFT� of g�n1 ,n2� is a sheet impulse function when itis a straight line of length N. From Eqs. �16� and �17�, weconclude that linear features are best extracted when they

Fig. 3 Fourier transform of image shown in Fig. 2.

run through the whole image. The shorter such features are

Apr–Jun 2006/Vol. 15(2)3



Downloaded

relative to the size of the image, the more their DFTs arespread out, rendering effective extraction difficult.

We provide a graphical illustration of the above argu-ment. Figure 2 shows a line of length �i.e., M� in a 100�100 �i.e., N=100� image. Figures 3 and 4 show the DFTG�k1 ,k2� of this image. The sine over sine nature is clearlyseen. Figure 5 shows that the polar transform of the Fouriertransform results in a spread of values in the polar domain.When a Fourier transform is performed on the polar trans-form, the line is not transformed to a delta function �Figs. 6and 7�. Thus, the Fourier transform approach will fail inseparating short lines due to the reduced SNR for line tobackground texture.

4 Handling Lines of Varying LengthsFourier transformation is the key ingredient of the BL ap-proach. Its properties of mapping straight lines to straightlines and isotropic textures to rotationally symmetric com-ponents are exploited. We now propose ways of exploitingthese properties when the lines are not of full length.

Fig. 4 Surface plot of the Fourier transform of image in Fig. 2.

Fig. 5 Polar transform of the Fourier transform of the image in Fig.
2.


4.1 Fourier Transform Block ApproachAs shown in the earlier section, the Fourier transform of ashort line in an image is a 2-D sine over sine function. TheFourier transform of a straight line fitting the entire imageis a delta function. In an image having varied lengths andthicknesses, it would seem that when the size of a block ismade to fit the length of a line, it results in delta functionsthat can be removed by mask filtering on going through theother transformations. Hence, if the image can be split intoperfectly fit blocks, a better separation of linear features canpotentially be achieved. As we demonstrate later with anapplication, this approach brings with it blocking artifacts.Furthermore, for any block size, there are bound to be linesthat yet again do not fill the entire block. We thus offeranother approach that, being based on the wavelet trans-form, uses varying window sizes.

4.2 Best Tree Wavelet Packet DecompositionApproach

To solve the problem of best splitting, we can take advan-tage of certain properties of wavelet packet transforms. Inthe wavelet packet transform of images, edges in the imageshow up prominently at all levels of the wavelet packet

Fig. 6 Fourier transform of the polar transform of the image in Fig.2.

Fig. 7 Surface plot of image in Fig. 6.

Apr–Jun 2006/Vol. 15(2)4



Downloaded

transform.8 On the other hand, random texture in the origi-nal image takes the appearance of noise in the detail coef-ficients. One can therefore attempt to extract edges at alllevels of detail from the background noise. Hence, liftinglinear features in the detail and approximation coefficientsof the wavelet packet transform against the noisy back-ground contributed by the random texture will likely resultin better separation of linear features from the texture.However, unlike the Fourier transform, the wavelet trans-form cannot separate lines from background texture by it-self. Therefore, we propose an approach that first performsa wavelet packet transform �WPT� followed by a Fourier-based line extraction at several levels, finally followed byan inverse wavelet packet transform �IWPT�.

The basic approach is outlined in Fig. 8. First, we per-form a wavelet packet decomposition of the input image.For each final subband, the BL approach is applied to createa “lines” image and a “background” image. The “lines”subband images and the “background” subband images areseparately inverse wavelet transformed to yield the finallines and background images. The best wavelet tree �BWT�approach is used to obtain the decomposition and recon-struction trees. In this approach, a root node n is split intoits lower subnodes only if the sum entropy of the resultingsubbands is lower than the image entropy at node n.MATLABTM utility besttree determines the entropy at anylevel using the modified formula9,10

E�s� = − �i

si2 log �si

2� , �18�

where E�s� is the entropy, and si2 is the normalized gray

value of pixel i. The normalization is performed such that�isi

2=1.

Fig. 8 Wavelet packe

Fig. 9 Block diagram of Wavele



A question that arises is to how many levels of waveletdecomposition one should perform with the best tree de-composition to achieve good line extraction. The larger thenumber of levels, the better we are able to extract smallerlines. However, this would result in really small linear fea-tures in the background also being extracted. On the otherhand, the fewer the number of levels, the less we take ad-vantage of the wavelet transform and the more likely thatwe achieve a performance commensurate with the BLmethod.

We propose using the entropy metric to determine thenumber of levels as well. The rationale is as follows. Whenthe number of decomposition levels is small, only thelonger lines in the original image yield linear features at thedetail levels that run through the entire detail blocks. Con-sequently, these are the features that are extracted when theBL approach is tried at the detail levels. As the number oflevels increases, shorter lines are extracted as well. Thus,with the increasing detail, the entropy of the reconstructed“lines-only” image must eventually increase with an in-creasing number of levels. One can stop when the entropyreaches a predetermined fraction �such as 90%� of the en-tropy of the input image. This process is shown in Fig. 9.

5 ResultsThe results of the best tree wavelet packet �BWP�, block-based Fourier transform �BFT�, and the BL approaches arepresented. The images were selected so that they all pos-sessed linear features against a background texture. TheDaubechies wavelet of order 8 is used for the BWP ap-proach. The first image �Fig. 10� is a Brodatz texture�“bark”� with linear features that run the entire length of theimage. The BL approach would be expected to be effective

extraction approach.
t line
t-based block technique.

Apr–Jun 2006/Vol. 15(2)5



Downloaded

given that the lines run all the way through. Figure 10shows the original along with the reconstructed line andbackground images of the three methods. The second im-age is a tiled floor with a mix of grooves and lines, somerunning throughout the image and others confined tosmaller stretches. Figure 11 shows the original and the re-constructions. Finally, a third image was generated by add-ing an image containing two straight lines of differinglengths �Fig. 12� to an artificially synthesized texture back-ground. This was done so that a measure could be obtainedto quantify the degree of line extraction. Figure 12 showsthe original and the corresponding reconstructions. The re-sults are analyzed in the next section.

Fig. 10 Bark image. Top, original image; middle �left to right�, lineslifted by the BL, BFT, and BWP, respectively, and bottom �left toright�, background from BL, BFT, and BWP, respectively.

Fig. 11 Floor tile image. Top, original image; middle �left to right�,lines lifted by BL, BFT, and BWP, respectively; and bottom �left to
right�, background from BL, BFT, and BWP, respectively.


6 Conclusion

By visual inspection of Fig. 10, we see that all three meth-ods perform equally well in separating the linear featuresfrom the background texture. This is to be expected sincethe lines run through the entire image. However, the BWPapproach outperforms the others in retaining fine texturalfeatures of the background in the background image, whilethe BFT approach clearly shows blocking artifacts arisingfrom the inherent block formation involved in the ap-proach. Figure 11 shows the ability of the BWP approach tolift finer linear features relative to the BL approach. This isespecially evident with respect to small lines near the cor-ners of the image. Consequently, the line reconstructionfrom the BWP is crisper. At the same time, the BWP ap-proach is more effective at leaving background point struc-tures in the background image with less distortion than theother methods. Again, the BFT approach is marred by theblocking artifacts. To compare these three methods with aquantitative measure, the peak signal-to-noise ratio �PSNR�was calculated for Fig. 12 for the different approaches. ThePSNR was defined as

PSNR = 20 * log 10�b/rms� , �19�

where b is the largest value of the signal, and rms is theroot mean square difference between the reconstructed lineimage and the original line image. The PSNR results areconsistent with our conclusions made above from visualinspection. The results were as follows:

1. The PSNR result for the BL approach is 13.95 dB.2. The PSNR result for the BFT approach is 6.02 dB.3. The PSNR result for the BWP approach is

Fig. 12 Artificially synthesized image. Top �left to right�, original ar-tificially synthesized image, random lines template, and synthesizedbackground; middle �left to right�, lines lifted by BL, BFT, and BWP,respectively; and bottom �left to right�, background from BL, BFT,and BWP, respectively.

12.64 dB

Apr–Jun 2006/Vol. 15(2)6



Downloaded

By this measure, the line extraction from the BWP methodoffers an improvement of over a decibel over the BLmethod for this example.

Overall, the BWP approach combines the advantage ofmultiscale identification of line features with the ability ofthe Fourier-transform-based BL method to actually separateline and isotropic background images to provide an en-hanced method of effecting line and background separation.One can control the scale of the features that fall into theline image or the background image by varying the depth towhich the best tree wavelet packet decomposition is per-formed. The BWP approach also provides an interestingillustration of the fact that Fourier and wavelet techniquescan work together and complement each other in solving aproblem. This is in contrast to most work in the early pe-riod of wavelet transform development that offered wavelettransforms as a better replacement for Fourier-type tech-niques in solving signal and image processing problems.

AcknowledgmentThis work was supported in part by Xerox-UAC Grant HE-1442-2003.

References1. J. Beyerer and F. P. León, “Adaptive separation of random lines and

background,” Opt. Eng. 37�10�, 2733–2741 �1998�.2. J. Campos and T. Kasparis, “Classification of periodic patterns using

the Hough transform,” in Proc. IEEE Southeastcon, pp. 367–371�1994�.

3. J.-C. Chien and C. C. Li, “Wavelet-based line detection in gray scaleimages,” in Proc. IEEE Int. Conf. on Systems, Man, and Cybernetics,Vol. 4, pp. 3670–3673 �1997�.

4. J. Hou and R. H. Bamberger, “Orientation selectiveoperators forridge, valley, edge, and line detection in imagery,” in Proc. IEEE Int.Conf. on Acoustics, Speech, and Signal Processing, Vol. 5, pp. 25–28�1994�.

5. J. Beyerer and F. P. León, “Detection of defects in groove textured ofhoned surfaces,” Opt. Eng., 36�1�, 85–93 �1997�.

6. V. Setlur and B. Gooch, “Visual perception and communication: isthat a smile? gaze dependent facial expressions,” in Proc. 3rd Int.



Symp. Non-Photorealistic Animation and Rendering, pp. 79–151,ACM Press, New York �2004�.

7. D. Dudgeon and R. Mersereau, Multidimensional Digital Signal Pro-cessing, Prentice-Hall, Englewood Cliffs, NJ �1984�.

8. R. M. Rao and A. S. Bopardikar, Wavelet Transforms—Introductionto Theory and Application, Addison Wesley Longman, Reading, MA�1998�.

9. http://www.mathworks.com/access/helpdesk/help/toolbox/wavelet/wavelet.html, MATLAB Wavelet Toolbox Documentation, The Math-works Inc., Natick, MA �2005�.

10. A. Jensen and A. la Cour-Harbo, Ripples in Mathematics, SpringerVerlag, Berlin �2001�.

Bhavani Pidaparthi: Biography and photograph not available.

Raghuveer M. Rao received his ME de-gree in electrical communication engineer-ing from the Indian Institute of Science in1981 and his Ph.D. degree in electrical en-gineering from the University of Connecti-cut, Storrs, in 1984. After serving as amember of technical staff at Advanced Mi-cro Devices, Inc., from 1985 to 1987, hejoined the Rochester Institute of Technol-ogy, New York, where he is a professor ofelectrical engineering and imaging science.

He has held visiting appointments with the Indian Institute of Sci-ence, the Naval Surface Warfare Center, the Air Force ResearchLaboratories, and Princeton University. Dr. Rao was an associateeditor for the IEEE Transactions on Signal Processing and the IEEETransactions on Circuits and Systems—II and is currently an asso-ciate editor for the Journal for Electronic Imaging. He received theIEEE Signal Processing Society’s Best Young Author Paper Awardand is a fellow of SPIE.

Honghong Peng received his MS degreein optical engineering in 2001 from theBeijing Institute of Technology, China, andsince October 2004, has been a PhD can-didate with the Center of Image Science ofthe Rochester Institute of Technology. Hisresearch interests include image process-ing, video processing, and pattern recogni-tion.

Apr–Jun 2006/Vol. 15(2)7


Documents

Separating linear features from textured backgrounds