Wavelet Analysis for Shadow Detection in Fringe

Projection Profilometry Ahmad Fadzil M Hani

1, Arwan Ahmad Khoiruddin

2, Nicolas Walter

1, Ibrahima Faye

1

1Centre for Intelligent Signal and Imaging Research, Universiti Teknologi PETRONAS

Bandar Seri Iskandar, Perak, Malaysia 31750 2Department of Informatics, Universitas Islam Indonesia

Yogyakarta, Indonesia 55584

Email: fadzmo@petronas.com.my

Abstract— Fringe Projection Profilometry (FPP) is the most

widely used method for 3D reconstruction. In FPP, the elevation

information is deduced from the deformation of the pattern

induced by the object. In the shadow areas, no fringe patterns

are formed and as a result, the elevation information in these

areas cannot be determined resulting in errors in the 3D

reconstruction. This paper proposes a new method to detect the

occurrences of shadow using the Haar Wavelet for a typical

single camera with one projector configuration. The Haar

wavelet emphasises the sudden change caused by boundary of the

shadow and also shows the non-existence of the pattern in the

shadow area. The method detects shadow boundary and

differentiates shadow and non-shadow area based on the

threshold defined from the statistics of the signal. The method is

found more suitable for deformed fringe pattern than Fourier-

based shadow detection, as the nature of the signal is non-

stationary. The proposed method correctly detects shadow with

robustness to the noise power up to 16.9 dB.

Keywords: Fringe Projection Profilometry, 3D Reconstruction,

Shadow detection, Shadow removal, Haar Wavelet

I. INTRODUCTION

Fringe Projection Profilometry (FPP) for 3D reconstruction has been used in many applications such as measuring the roughness of psoriasis lesions [1], biometric identification [2], industry quality control [3–6], etc. FPP typically comprises of a camera and a projector. The projector projects a fringe pattern onto a scene (object) while the camera captures the illuminated scene. Due to the object surface profile, the fringe pattern captured by the camera is deformed. By performing line-by-line analysis of the fringe deformation, surface elevations can be deduced. The principles of surface reconstruction for FPP are detailed in our previous work [7].

It is known that fringe patterns are not formed in areas that are occluded by other protruding surfaces causing shadows in these areas [8,9]. In the presence of shadow, no patterns are available and thus, the elevation information of the surface is lost. Without localizing the shadow area, reconstruction errors will occur. Several methods have been proposed to detect shadow [8–13]. Prati, et al [10] and Sanin, et al [11] reviewed some methods used for moving cast shadow. However, in our FPP application, the object and the shadow are not moving. To detect shadow from non-moving image, several methods have been proposed. Bringier, et al [13] used photometric stereo to detect and remove shadow and specular problem. Boleček, et

al [12] used a general assumption that shadow pixels have lower intensity than pixels illuminated by the fringe pattern. To avoid false detection caused by low intensity from dark object, depth information of the scene acquired from photometric stereo is inputted. Nevertheless, these methods need two cameras while in our work, the classical FPP configuration with one-camera one/multi-projector is used. Shadow detection method using one-camera multi-projector configuration has been proposed by Skydan et al [8, 9]. To detect the invalid area caused by shadow, the inverse Fourier Transform method is used. The method generates masks that identify the invalid areas caused by shadows for each projector. The mask and 3D reconstruction obtained from each projector are combined to achieve a shadow free surface profile. However, due to the complexity of the object, shadow areas can remain. Moreover, the nature of the deformed pattern is non-stationary while Fourier Transform is more suitable for stationary signal, leading to inaccuracies in the 3D reconstruction [14–17].

This paper proposes a new method for detecting shadow in FPP based on wavelet analysis. In wavelet spectrum, shadow boundary and sharp edges of the inspected object will induce peaks. As no signal is detected in shadow areas, wavelet spectrum is equal to zero while in non-shadow areas, the wavelet spectrum oscillates due to the nature of the sinusoidal pattern. This phenomenon will allow the shadow areas to be distinguishable. In the presence of noise, the wavelet spectrum in shadow areas is not zero even if the noise is filtered. Nevertheless, by detecting the peaks and using wavelet spectrum statistical features between the peaks (mean and standard deviation), shadow areas can be determined.

This paper is organized as follows. Section II discusses the mathematical model of FPP and shadow for the simulations. Section III provides the details of the proposed method. Section IV analyses the performance of the method and its robustness to noise. Section V concludes the paper.

II. SYSTEM MODEL

The schematic diagram of a typical FPP using one camera and one projector is depicted in Figure 1. The projector projects fringe patterns onto the objects and the camera captures the illuminated scene. The projected pattern will be deformed due to the surface profile of the object. The elevation information is deduced by analysing the deformation

2012 IEEE Symposium on Industrial Electronics and Applications (ISIEA2012), September 23-26, 2012, Bandung, Indonesia

978-1-4673-3005-3/12/$31.00 ©2011 IEEE 336

of the fringe pattern. The analysis is performed line-by-line as in [15, 18, 19] or by analysing the whole image [12, 13, 17, 20].

In the presence of an object, a ray projected onto point J will be reflected to the camera as if it is reflected from point K. The elevation is formulated as

Figure 1 Schematic Diagram of FPP

Note that h(x) is the elevation of point G and is the ray displacement at point G. l is the distance from projector (and camera) to the reference plane and d is the distance from camera to projector.

Due to the projection angle, , the shadow problem in FPP is recurrent [8], [18–20]. In Figure 1, the shadow area due to the object occluding the fringe pattern in the FOV (red line) of

the camera is . It is seen that no fringe pattern is present in this area. However, the fringe pattern projected onto

continues in . With the shadow area determined, the remaining signal can be reconstructed appropriately thereby reducing reconstruction errors.

For simplicity, a ground truth of shadow area is generated

using the assumption that the inspected pixel on the object, the

camera, and the projector are always in one line. Figure 1

presents a sectional view of the acquisition system. In this

plane, the shadow ( can be formulated as follows:

Based on geometry and Thales theorem, the following equations can be deduced:

Using Equations (2), (3) and (4), the shadow area ( can be deduced as follows:

Figure 2 Block Diagram of the Proposed Algorithm

From Equation (5), it can be seen that the shadow is

influenced by the elevation difference ( ) and the

configuration of the acquisition system (d and l). The shadow

is proportional to the distance camera-projector (d) or

projection angle, .

III. WAVELET ANALYSIS FOR SHADOW DETECTION

To detect the shadow in the image, the analysis is

performed line-by-line. Our method can be described by the

block diagram shown in Figure 2. Due to imperfections of

camera and projector, the presence of noise can make the

shadow boundary unclear. A median filter is applied as a pre-

processing function to reduce the noise while preserving the

shadow boundary. Deformed patterns in FPP are non-stationary signals [15].

Wavelet Transform has been successfully used in analysing non-stationary signal [14–17]. Since the wavelet transform gives both time and frequency domain analysis, it will have the information about the position of the shadow. As seen in Figure 3, the shadow boundary (at x=279 to x=299) creates a sudden signal transition. Note that the surface elevation at H (at x=701) causes a phase change in the sinusoidal fringe pattern. Haar wavelet is used here as its shape is the most suitable to detect sudden signal transition [21].

The Haar mother wavelet function can be described as

2012 IEEE Symposium on Industrial Electronics and Applications (ISIEA2012), September 23-26, 2012, Bandung, Indonesia

337

Figure 3 Deformed signal of the middle of the camera

Field of View

Figure 4 The wavelet transform coefficients (gray) overlaid

with the deformed signal (green) and the wavelet

coefficient of the first scale (red).

Its scaling function is formulated as

Figure 4 shows the wavelet transform coefficients of the deformed signal (overlaid in Figure 3). At the boundary of the shadow where the phase change occurs, the wavelet coefficients have high values (hence the brightness in the wavelet transform coefficient diagram). By observing the wavelet coefficients at scale=1 (corresponding to high frequency- Figure 5), it is clear that in the shadow area, the wavelet coefficients are zero while in the non-shadow areas, the wavelet coefficients oscillate between zero and some value. Therefore, by observing the behaviour of wavelet coefficients between peaks, a shadow mask can be created.

However, in the presence of noise (Figure 6), it is difficult

to differentiate between the shadow and non-shadow areas as

the noise makes the shadow areas to have a non-zero random

pattern. To remove the noise, the image is filtered by a

median filter. Median filter is a non-linear filter that is

effective in removing impulsive noise [22] without

delocalising (blurring) the edges in the image. To retain the

edge, 1-D median filter order 3 is used. The result after

applying the median filter is shown in Figure 7.The result

shows that the noise has been reduced yet retaining the edge.

The next step is to determine the peaks in the wavelet

coefficients. This can be achieved by a thresholding process

based on the statistics (mean and standard deviation) of the

wavelet coefficients of the filtered signal. The peaks are

coefficients having values greater than a threshold formulated

as follows:

Note that and are the mean and the standard deviation of

the absolute power of wavelet coefficients, respectively.

Once the peaks are detected, we have i-number of area

between peaks. These areas are then classified as shadow or

non-shadow areas. The mask of shadow/non-shadow areas is

formulated with the following mathematical model:

is the mask of the area. 0 is the mask value for shadow

and 1 is the mask value for non-shadow area. is the

mean of area i while is the standard deviation of the

area i. and are the mean and standard deviation of the

wavelet coefficient in non-peak value.

Figure 5 Wavelet coefficients at scale =1

Figure 6 Wavelet coefficients at scale=1 with additive

white Gaussian noise 16.9 dB (

2012 IEEE Symposium on Industrial Electronics and Applications (ISIEA2012), September 23-26, 2012, Bandung, Indonesia

338

Figure 7 Haar wavelet of the filtered signal using median

filter

IV. SIMULATION

A rectangular shape object is used in this simulation as shown in Figure 8. The configuration of the system used for the simulation is as follows: The distance between the camera and projector (d) is 2000 mm while the distance between the projector and camera to the reference plane is 5000 mm. The dimensions of the object are 400 mm (width), 400 mm (length) and 160 mm (height). The image is of size 1000x1000 pixels with a resolution of 1mm/pixel. The spatial frequency of the fringe pattern (f) used is 10 Hz. The image of the fringe pattern projected onto the object is shown in Figure 9. The signal is generated using the following formula:

where and are the intensity and the phase of the fringe at coordinate x, respectively. Equation (5) is used to generate shadow on the left border of the object (due to the configuration described in Figure 1).

In order to evaluate the technique, its robustness to noise is studied. White Gaussian noise with different standard deviation (power) is tested on the simulated object.

To measure the error of our method objectively, the ground truth of the shadow mask is generated using Equation (5). The standard deviation of the White Gaussian noise is varied from 1 (0 dB) to 20 (26 dB). For each level of noise, 30 experiments are performed. For each experiment, different random White Gaussian Noise is added. The mean and standard deviation of the masking error are shown in Figure 10. From Figure 10, it is found that the algorithm gives perfect shadow detection if the standard deviation of the noise is less than or equal to 7 (16.9 dB).

Figure 8 Rectangular Object used in the simulation

Figure 9 Rectangular Object Projected with Fringe

Pattern.

Figure 10 Error of the shadow masking

Figure 11 Detection Rate (DR) of the proposed algorithm

Two metrics can be used for the evaluation of the detection

of shadow, i.e. Detection Rate (DR) and False Alarm Rate (FAR) [10]. The metrics are defined as follows:

Note that TP, FN and FP are True Positive, False Negative and False Positive, respectively.

2012 IEEE Symposium on Industrial Electronics and Applications (ISIEA2012), September 23-26, 2012, Bandung, Indonesia

339

Figure 12 False Alarm Rate (FAR) of the proposed

algorithm

On one hand, the results in Figure 11 show that False Negative errors occur if the standard deviation of the noise is greater than 8 (18 dB). On the other hand, Figure 12 shows that False Positive errors occur if the standard deviation of the noise is more than 7 (16.9 dB). Thus, to ensure that all shadow parts are detected without error, the equivalent noise power should be less than 16.9 dB (SD < 7).

V. CONCLUSION

In this paper, a new method for detecting shadow in FPP using wavelet has been proposed. The performance of the method has been evaluated by studying its robustness to noise. The method is robust to additive white Gaussian noise with standard deviation up to 7 (16.9 dB). No False Negative and Positive errors are exhibited.

The mask created from our method is used for localizing the shadow area during phase unwrapping. The area masked by the shadow will not be unwrapped and reconstructed. In FPP with one-camera multi-projection configuration, this mask can be used for combining the reconstruction results i.e. replacing the shadow area resulting from one projector with non-shadow area from another projector.

ACKNOWLEDGMENT

This work was supported by ViTrox Corporation Berhad

under MDEC Grant no 15-8500-039.

REFERENCES

[1] M. H. A. Fadzil, E. Prakasa, H. Fitriyah, H. Nugroho, A. M. Affandi,

and S. H. Hussein, “Validation on 3D Surface Roughness Algorithm for Measuring Roughness of Psoriasis Lesion,” Engineering and

Technology, no. 1, pp. 116-121, 2010.

[2] B. Redman et al., “Stand-off Biometric Identification using Fourier Transform Profilometry for 2D+3D Face Imaging,” Applications of

Lasers for Sensing and Free Space Communications, OSA Technical

Digest (CD) (Optical Society of America, 2011), paper LThB3., pp. 3-5, 2011.

[3] T.-W. Hui and G. K.-H. Pang, “3D profile reconstruction of solder

paste based on phase shift profilometry,” IEEE International Conference on Industrial Informatics, pp. 165-170, 2007.

[4] H.-nan Yen, D.-ming Tsai, and J.-yi Yang, “Full-field 3D measurement

of solder pastes using LCD-based phase shifting techniques,” IEEE Transactions on Electronics Packaging Manufacturing,, vol. 29, no. 1,

pp. 50-57, 2006.

[5] K. M. Jeong, J. Seon, K. Kyoung, C. Koh, and H. S. Choc,

“Development of PMP system for high speed measurement of solder paste volume on printed circuit boards,” Proc SPIE Optomecatronic

Systems, vol. 4564, no. 2001, pp. 250-259, 2001.

[6] B. Luo, “SMT solder paste deposit inspection based on 3D PMP and 2D image features fusion,” in Proceedings of the 2010 International

Conference on Wavelet Analysis and Pattern Recognition, 2010, no.

July, pp. 190-194. [7] A. F. M. Hani, A. A. Khoiruddin, N. Walter, I. Faye, and T. C. Mun,

“3D Reconstruction using Spline Inverse Function Analysis,” in

International Conference on Intelligent & Advanced Systems, 2012, pp. 479-482.

[8] O. A. Skydan, M. J. Lalor, and D. R. Burton, “New technique for phase

measurement and surface reconstruction using coloured structured light,” Engineering, 2005.

[9] O. Skydan, M. Lalor, and D. Burton, “Using coloured structured light

in 3-D surface measurement,” Optics and Lasers in Engineering, vol. 43, no. 7, pp. 801-814, Jul. 2005.

[10] A. Prati, I. Mikic, M. Trivedi, and R. Cucchiara, “Detecting Moving

Shadows: Algorithms and Evaluations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 918-923, 2003.

[11] A. Sanin, C. Sanderson, and B. C. Lovell, “Shadow detection: A survey

and comparative evaluation of recent methods,” Pattern Recognition, vol. 45, no. 4, pp. 1684-1695, Apr. 2012.

[12] L. Boleček and V. Říčný, “MATLAB DETECTION OF SHADOWS

IN IMAGE OF PROFILOMETRY,” in Technical Computing Prague 2011, 2011, no. 1, pp. 22-30.

[13] B. Bringier, A. Bony, and M. Khoudeir, “Specularity and shadow detection for the multisource photometric reconstruction of a textured

surface.,” Journal of the Optical Society of America. A, Optics, image

science, and vision, vol. 29, no. 1, pp. 11-21, Jan. 2012. [14] A. Dursun, N. Ecevit, and S. Ozder, “Application of wavelet and

Fourier transforms for the determination of phase and three-

dimensional profile,” in Proceedings of International Conference on Signal Processing, Int. J. Comput. Intell. 1, 2003, pp. 168-172.

[15] A. Z. Abid, M. a. Gdeisat, D. R. Burton, M. J. Lalor, H. S. Abdul-

Rahman, and F. Lilley, “Fringe pattern analysis using a one-dimensional modified Morlet continuous wavelet transform,”

Proceedings of SPIE, vol. 7000, p. 70000Q-70000Q-6, 2008.

[16] S. Fernandez, M. a. Gdeisat, J. Salvi, and D. Burton, “Automatic window size selection in Windowed Fourier Transform for 3D

reconstruction using adapted mother wavelets,” Optics

Communications, vol. 284, no. 12, pp. 2797-2807, Jun. 2011. [17] M. a Gdeisat, D. R. Burton, and M. J. Lalor, “Spatial carrier fringe

pattern demodulation by use of a two-dimensional continuous wavelet

transform.,” Applied optics, vol. 45, no. 34, pp. 8722-32, Dec. 2006. [18] Y. Cai and X. Su, “Inverse projected-fringe technique based on multi

projectors,” Optics and Lasers in Engineering, vol. 45, no. 10, pp.

1028-1034, Oct. 2007. [19] L. Chen, C. Quan, C. Jui Tay, and Y. Huang, “Fringe contrast-based 3D

profilometry using fringe projection,” Optik - International Journal for

Light and Electron Optics, vol. 116, no. 3, pp. 123-128, Apr. 2005. [20] F. Sadlo, T. Weyrich, R. Peikert, and M. Gross, “A Practical Structured

Light Acquisition System for Point-Based Geometry and Texture,”

Eurographics Symposium on Point-Based Graphics, 2005. [21] B. Y. Lee and Y. S. Tarng, “Application of the Discrete Wavelet

Transform to the Monitoring of Tool Failure in End Milling Using the

Spindle Motor Current,” International Journal of Advanced Manufacturing Technology, vol. 15, pp. 238-243, 1999.

[22] T.-wai Hui and G. K.-hung Pang, “3-D Measurement of Solder Paste

Using Two-Step Phase Shift Profilometry,” IEEE Transactions on Electronics Packaging Manufacturing, vol. 31, no. 4, pp. 306-315,

2008.

2012 IEEE Symposium on Industrial Electronics and Applications (ISIEA2012), September 23-26, 2012, Bandung, Indonesia

340