The importance of phase in image Processing

Original Lena Image

Iterative schemes-Localized RLPQ edges preservingGlobal RLPQ

IntroductionPhase is an important signal component, which is often ignored in favor of magnitude.Phase is sufficient for image segmentation, edges detection etcPhase manipulations result in various useful effects. Common image spectraNatural Images statistical average spectrum[1]

Lena image spectrumWhere is the data encoded?

2D Fourier magnitude2D Fourier phaseAmlitude is very structured, and common between images, as opposed to phase which has uniform noise/ salt and pepper like nature.5The importance of phase in images [2]

Texture mosaics images

Voice STFT spectrogram

The importance of phase in voice

STFT?Image reconstruction from phase [3].

Global vs. Local schemes [4]SlidingwindowShort Time Fourier Transform

...

FT

2D PC2D ASNo unique definition for the multidimensional HT or AS exists.

Most general 2D HT definition is radial HT- H(w1,w2)=() (sufficient for edges detection).

For now, 2D PC is defined via combination of several 1D PCs in different orientations.

A truly multidimensional PC?Verify Logans condition34Image segmentation- Gabor Filters

[5-7]Image segmentation- Gabor Wavelets

Image segmentation- Filtering results

Image segmentation- Gabor feature space

Magnitude based feature spacePhase based feature space

Image segmentation- Clustering

K-means ClusteringSegmentation with phase[6,7]Feature space: Gabor phase response difference statistics:

Brodatz Mosaics segmentation[5]

Magnitude onlyTested mosaicPhase only[6]

Phase only[7]Phase & Magnitude[6]Phase & Magnitude[7]

How?Our implementation- unsupervised segmentation, using [5,6] feature space. In [5,6]- supervised learning (ML, MAP) is discussed.20Natural images segmentation[5]Magnitude onlyTested imagePhase only[6]Phase only[7]Phase & Magnitude[6]Phase & Magnitude[7]

All tests21Natural images

Segmentation results- tables

Texture mosaics resultsNatural images resultsTest imagesWhile the grades (% of correctly classified pixels) may be deceiving in a way (especially in the case of natural images), they state that the phase based feature space is reach enough, and thus achieved better results/ eliminates exhaustive parameter search, as opposed to magnitude based methods. 24Edge detection

Analytical Signal and Hilbert Transform

Logans condition for signal reconstruction from zero crossing- no common complex zeroes to the signal and its HT.AS is used in many applications: communication, radars, inverse filtering, image processing, speech processing, data compressing etc

26Phase Congruency (PC) based Edge detection

[7]

Even (cosine) and Odd (sine) components.Owens & Robis Local energy- for certain phase values. Kovesi for all congruent phase27PC Edge detectionIm{FT[x]}Re{FT[x]}-Freq. comp. 1-Freq. comp. 2-Freq. comp. 3-Freq. comp. 4Im{FT[x]}Re{FT[x]}-Freq. comp. 1-Freq. comp. 2-Freq. comp. 3-Freq. comp. 4

Im{FT[x]}Re{FT[x]}-Freq. comp. 1-Freq. comp. 2-Freq. comp. 3-Freq. comp. 4Im{FT[x]}Re{FT[x]}-Freq. comp. 1-Freq. comp. 2-Freq. comp. 3-Freq. comp. 4PC Edge detection (cont.)PC ? ASEdge detectors-1DOriginal SignalEdges via phase STDPC via

AS edge detection31

Edge detectors-1DOriginal SignalAS Energy, Local EnergySig. derivative

2D- PC?AS edge detection322D PC via 1D PCs projections

Anti noise

Edge detection- Localized Phase Quantization error (LPQe) scheme[9]Phase build edges, thus phase impairment ruins primary edges. Moreover, in case of cartoon image, inter edges areas DC or Low freq, and there phase has little influence. Was recently presented in EUSIPCO36LPQe edge detector-1DOriginal SignalLPQe

Edge detection, highly similar to gradient37Edge detectors-2DOriginal SignalPC|LPQe|

LMIe?Darker background?38Edge detectors- dealing with noiseOriginal SignalSNR 10[dB]PC|LPQe|Raw Canny [10]

Canny thresholdsNo parameters- canny can be improved by much with parameters changes, while CP and LPQe is virtually params free.40PC based application: Geodesic snakes segmentation[11]Snakes?

Repeat snakes! Why phase based edges are better? More robust (no units, bounded, etc). Make it work from Power point421D LPQe based application: P&M anisotropic diffusion

[12]Gradient like behavior- we can easily replace diff.44Man-mades detection via FractalsFractals are mathematical objects defined by B.B MandelbrotNatural objects usually self similarity Able to easily generate and represent natural-like shapes

Each part of the image has different fractal dimensions, -> feature space.

[13]Man-mades detection via FractalsGrayscale levels represent the height of the surface.

The area is measured at different scales to check the fractal model fit, according to:

The 3 fractal model parameters are calculated for each pixel: the fractal Dimension D, the constant F and the model fit error.

2D LPQe based application: Detection of Man-Made environmentGray scale imageLPQe edges mapFractals?[13]

PC edges map

Sometimes, not all signal information is needed. In this case edges map, is sufficient for man mades detection. Edges map carries less data, and thus easier to store, transmit and analyze.47Phase quantization- how to, ?

Phase quantization- how to, ?

Phase quantization- how to, ?

When quantized phase reconstruction is real?Real

Thus to result in a real signal the Quantization method Q must be anti-symmetric:

Rotated Local Phase QuantizationOnly asymmetric quantization scheme results in a non complex signal.

Therefore the Rotated Quantization scheme resulting signal is complex for all values

Meaningful Real and Imaginary components

Proof52Rotated Local Phase QuantizationImaginary{RLPQ}- blurred signal.

Blurring effect very similar to Box Blur.

Blur from Im{RLPQ}

Edges from Re{RLPQ} Kq=2

Cartoons from Re{RLPQ} Kq=3

Image primitives from Re{RLPQ}Kq>>2Kq=3Kq=2Edges MapCartoonOriginal imageLocalized KqEdges carry information, thus preserving edges during RLPQ is vital.Means localized, signal dependent Kq!

||LPQe||KqInput imageEdges DetectionSignal dependent RLPQ

TeD like results

Diffusion like results via RLPQ

OrigRLPQHeat Diffusion [14]

TeD and edge preserving RLPQ

RLPQTelegraph Diffusion [15]

Iterative RLPQConclusionsWe have shown that use phase can replace magnitude in various algorithms (segmentation, edges detection, etc) and sometimes result in a better performance.We have shown that common signal/image processing tasks such as: HP filtering and can be achieved via localized phase manipulations.Our RLPQ output (simultaneous cartoonization and edge detection) visually similar to results achieved by diffusion schemes (P&M, G. Gilboa FaB, V. Ratner TeD).ReferencesA. V. Oppenheim, and J. S. Lim, The importance of phase in signals, Proceedings of the IEEE, vol. 69, no. 5, pp. 529-541, 1981.A. Torralba, and A. Oliva, Statistics of natural image categories, Network, vol. 14, no. 3, pp. 391-412, Aug, 2003.J. Behar, M. Porat, and Y.Y. Zeevi. Image reconstruction from localized phase, IEEE Transactions on Signal Processing, Vol. 40, No. 4, pp. 736743, 1992.G. Michael, and M. Porat, "Image reconstruction from localized Fourier magnitude," Proceedings 2001 International Conference on Image Processing. pp. 213-16.A. C. Bovik, M. Clark, and W. S. Geisler, Multichannel texture analysis using localized spatial filters, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 12, no. 1, pp. 55-73, 1990.A. K. Jain, and F. Farrokhnia, Unsupervised texture segmentation using Gabor filters, Pattern Recognition, vol. 24, no. 12, pp. 1167-1186, 1991.H. Tanaka, Y. Yoshida, K. Fukami et al., Texture segmentation using amplitude and phase information of Gabor filters, Electronics and Communications in Japan, Part 3 (Fundamental Electronic Science), vol. 87, no. 4, pp. 66-79, 2004.A. P. N. Vo, S. Oraintara, and T. T. Nguyen, "Using phase and magnitude information of the complex directional filter bank for texture image retrieval," Proceedings 2007 IEEE International Conference on Image Processing, ICIP 2007. pp. 61-4.ReferencesJ. A. Davis, D. E. McNamara, D. M. Cottrell et al., Image processing with the radial Hilbert transform: theory and experiments, Optics Letters, vol. 25, no. 2, pp. 99-101, 2000.N. Skarbnik, C. Sagiv, and Y. Y. Zeevi, "Edge Detection and Skeletonization using Quantized Localized phase," Proceedings of 2009 European Signal Processing Conference, EUSIPCO-09.M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active contour models." International Journal of Computer Vision. v. 1, n. 4, pp. 321-331, 1987. P. Perona, and J. Malik, Scale-Space and Edge Detection Using Anisotropic Diffusion, IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 7, pp. 629-639, 1990.M. J. Carlotto, and M. C. Stein, A method for searching for artificial objects on planetary surfaces, Journal of the British Interplanetary Society, vol. 43, no. 5, pp. 209-16, 1990.V. Ratner, Y. Y. Zeevi, and Ieee, "Telegraph-diffusion operator for image enhancement." IEEE International Conference on Image Processing (ICIP 2007), pp. 525-528, 2007.FinThank for your attention.Questions?

Refs.66