1

23rd World Gas Conference, Amsterdam 2006

WELDING DEFECT PATTERN RECOGNITION IN RADIOGRAPHIC IMAGES OF GAS PIPELINES USING ADAPTIVE FEATURE

EXTRACTION METHOD AND NEURAL NETWORK CLASSIFIER

Main author

S. MANSOURI ALGHALANDIS *

Co-author

GH. NOZAD ALAMDARI

* First Corresponding (Presenting) Author, Email: smansouria @ yahoo.com 1- Senior Gas Transmission Expert , National Iranian Gas Company (NIGC), R&D

Dept. , Dist-8 of Gas Transmission Operation, Tabriz 51745 - 367, Iran.

2- Weld Interpreting Expert , National Iranian Gas Company(NIGC), Mechanical Dept. , Dist-8 of Gas Transmission Operation, Tabriz 51745 - 367, Iran.

1

2

2

1. ABSTRACT: In this paper a new method for automatic recognition and separation between defected radiographic images of welds and correct ones is introduced. The method applies local image information together with adaptive feature extraction parameters and neural network classifier. In practical weld inspection of gas pipelines, more than sixty percent of radiographic images are not faulty, but need to be separated in visual check by a certified expert. This is a time consuming process for interpreter and reduces the eye sensitivity and reliability of inspection especially for defected parts of radiographic image. The introduced pattern classifier is designed to solve this problem. In first step, preprocessing is done through image enhancement and noise reduction techniques proper for poor quality and low contrast radiographic images. Preprocessing step is applied through median filtering and adaptive histogram equalization, together with preserving grayscale image information. In next step weld is extracted from background image and adaptive thresholding is applied using local binarization operators with maintaining critical image information. Then welded area is zoned based on the importance of the image information and features are extracted for different zoning patterns. A variety of topological features and also grayscale information related to each zone are used. Some film defects mistaken instead of weld defects are also detected in this stage. By using film defect information in classification stage, the recognition efficiency can be increased. The extracted features are fed into an artificial neural network classifier. A new structure of neural network classifier in combination with binary logic is introduced for classification stage. The reliability and accuracy of this new hybrid neuro-logic structure is compared with conventional neural networks according to adjusted local thresholding and adaptive zoning parameters. The problem of choosing appropriate features are also discussed and evaluated. The results show the recognition performance and flexibility of this new hybrid neuro-logic classifier in comparison with conventional neural networks structure. Key words: Radiographic pattern recognition Welding defect detection Adaptive feature extraction Neural network classifier

3

TABLE OF CONTENTS 1. Abstract 2. Introduction 3. Digitizing 4. Image enhancement and processing 4.1. Median filtering 4.2. Contrast enhancement 4.3. Wiener filtering 5. Image segmentation 5.1. Weld extraction 5.2. Local binarization 6. Feature extraction 6.1. Film defect feature 6.2. Unavailability of defected area 6.3. Average size of defects 6.4. Average distance of defects from center line 6.5. Number of defects 6.6. Zoning 7. Classification 7.1. Multi layer perceptron neural network architecture 7.2. New hybrid neuro-logic architecture 8. Results 8.1. Data 8.2. Recognition performance 8.3. Discussion 9. Conclusion 10. References 11. List of Tables 12. List of Figures

4

2. INTRODUCTION Radiography is one of the old and still effective NDT tools. The most important application of this method in gas industries is the inspection of the welds for transmission pipelines. Even with the invention of online digital radiography using sensitive fluorescent plates, offline analog radiography using films still has its own benefits and applications. Rapid development of gas distribution network reveals the necessity for a reliable and automatic weld inspection system. Checking poor quality and low contrast films of welds is a time consuming process for a certified expert and reduces his/her eye sensitivity and inspection accuracy especially for defected parts of radiographic images. The most of the papers published about automatic weld flaw recognition systems, all deal with identifying the type of the weld defects. This article deals with the separating the correct part of welding from its defected part in radiographic films. Since, more than sixty percent of radiographic images in gas pipelines are not faulty but need to be separated in visual check, our designed system can act as an effective tool to reduce the mass work of film checking process and put the concentration of the inspector on defected parts of the films. In the articles, there is no report about identification of film defects, which reduce the efficiency of weld flaw recognition system in classification stage. In this study a method is introduced for separation of film defects which can be mistaken instead of weld flaws, too. Extraction of welds from background image can be done automatically by computer or manually by user selection. There are various reported methodologies for automatic weld extraction based on the assumption that the intensity of the pixels in the weld area distribute more as Gaussian distribution than other areas in the image. An automatic weld extraction method is used in this study. Next step is image segmentation and feature extraction. In this article local thresholding is used for adaptive segmentation of welded area. Then some topological features are extracted from segmented area and fed into an MLP neural network classifier with back propagation algorithm. In this paper, an automatic weld defect separation system is designed and tested. At first, film digitizing is described. Then preprocessing and image enhancement is applied. Next stage local segmentation and feature extraction is studied. The extracted features together with film defect information are applied to an MLP neural network classifier. Finally a comparison between different sets of input parameters to pattern classifier is done and the results are discussed. 3. DIGITIZING Film digitizer as an entrance gate for image data input, is a critical part of the weld recognition system. Selecting optimized resolution of scanning and acceptable quality of digitizing plays an important role in whole system performance. Gamma-ray film strips were digitized using scanning device under controlled illumination with approximate film dimensions of 70 mm by 300 mm. A commercial scanner with light intensity of 20,000 cd./sq.m ( candle per square meter) was used. Different resolutions from 300 to 2000 dpi ( dot per inch) with gray scale ranges from 8 bit to 16 bit were tested to find an optimal selection. It is known that the human being is not able to differ in gray scale over 128 levels(7 bits) [1] and images of 16 bit depth in gray scale level occupy significant memory space and take long time to be processed. In figure 1, some scanned sample images with different resolution from 300 to 2000 dpi are shown. In this study for compromising between image quality and processing speed, images with resolution of 600 dpi and gray scale of 256 levels were selected.

(1.a)300 dpi. (1.b) 600 dpi. (1.c) 1600 dpi. (1.d) 2000 dpi. Fig.1. Digitized images with different resolution and 8 bits of gray level. (a)300 dpi. (b) 600 dpi. (c) 1600 dpi. (d) 2000 dpi.

5

4. IMAGE ENHANCEMENT AND PROCESSING Preprocessing and image enhancement is done to remove system noise and also photographic film noise in three steps including median filtering, contrast enhancement and wiener filtering. 4.1. Median filtering First of all, median filtering is applied through the image. For the mask shown in figure1, suppose that MED(i,j) is the median of the values in 5Χ5 neighborhood pixels where ( i , j ) represents pixel coordinates in the mask center of 13m . Filtered image is obtained by using the relation :

),,...,,,(),( 2524321 mmmmmMedianjiMED = (1)

The gray level of each pixel is replaced by median of the gray levels in the neighborhood of that pixel. This method is particularly effective for the noise patterns consisting strong, spike like components and where the characteristic to be preserved is edge sharpness [ 2]. Original image of weld and its 5Χ5 median filtered is shown in figure 2.

(2.a) (2.b) (2.c) Fig. 2. (a) 5Χ5 neighborhood pixels as a mask for median filtering in Eq.(1). (b) Original image (c) 5Χ5 median filtered image. 4.2. Contrast enhancement Different techniques of histogram equalization to enhance contrast of radiographic images have been reported in many researches [1],[3],[4],[5]. The enhancement technique of choice is the so called histogram equalization (HE). For discrete values of gray levels, probabilities given by the relation:

10)( ≤≤= kk

kr rn

nrP , k = 0, 1, … , L - 1 (2)

where kr represents the gray level of the pixels of the image to be enhanced, with kr = 0 representing

black and kr = 1 representing white in the gray scale, L is the number of levels, )( kr rP is the

probability of the k th gray level, kn is the number of times this level appears in the image. A plot of

)( kk rP versus kr is usually called histogram and the technique used for obtaining a uniform histogram

is known as HE technique [2]. In this technique the cumulative histogram H of gray levels G is used

as the essential part of the function ( )GFHE that maps the original gray levels into the transformed ones:

( ) minmaxmin

)(GGGwith

N

GHGGGFHE ′−′=′∆′∆+′= (3)

6

in which maxG′ and minG′ indicate the upper and lower limits of the transformed gray values,

respectively and N represents the number of pixels over which the histogram has been taken [6]. HE is a global approach based on gray level distribution over an entire image and is not suitable to enhance details over small areas. The global histogram equalization is made adaptive by taking the histogram over a local region instead of the whole image:

AHE

AHEAHE N

GHGGGF

)()( min ′∆+′= (4)

Adaptive histogram equalization (AHE) has been recognized as a valid method of contrast enhancement in image processing. The main advantage of AHE is that it can provide better contrast in local areas than that achievable utilizing traditional histogram equalization methods. Whereas traditional methods process the entire image at once, AHE utilizes local contextual region. The effect of AHE in contrast enhancement of this study is shown in figure 3.

(3.a) (3.c) (3.d)

(3.b) (3.e) Fig.3. (a) Original image. (b) Histogram of original image. (c) Adaptive histogram equalization(AHE), The contextual region shown is an m х m mask around a pixel at location (x,y). (d) Image after AHE (e) Histogram of image after AHE. 4.3. Wiener filtering Application of wiener (least-mean-square) filter in image restoration has been already reported as a linear filter [2]. In this study wiener filter is applied to the image adaptively, tailoring itself to the local Image variance. Where the variance is large, it performs little smoothing, where the variance is small it

7

performs more smoothing. This approach is more selective than comparable linear filter, preserving edges and other high-frequency parts of the image. It works best when noise is constant-power additive noise, such as Gaussian noise. Since the intensities of the pixels in the weld area distribute more as a Gaussian distribution than other areas in the image of weld [7], it was selected for filtering radiographic images of welds. 5. IMAGE SEGMENTATION The segmentation methodology includes the following steps: 5.1. Weld extraction Before doing any further local processing on the radiographic image, it is preferred to extract weld region by separating between welded area and its background image. For this reason, proposed methodology by Liao et al. [8], is applied in this research. 5.2. Local binarization Binarization of scanned gray scale images is an important step in most image analysis systems. For poor quality images such as radiographic films, global thresholding doesn’t have good performance for areas with variable background intensity, low contrast and stochastic noise. Therefor it is essential to find binarization methods which will correctly label all the information present. Different local binarization methods are reported by many researchers[9]. Niblack’s method is known to give the best performance for the context of digit recognition [10]. In this study, Niblack’s method is used by tuning its parameters for radiographic images of welds. The idea of this method is to vary the threshold over the image, based on the local mean and local standard deviation. The threshold at pixel ( x, y) is calculated as

),(.),(),( yxskyxmyxT += (5) Where m( x, y) and s( x, y) are the sample mean and standard deviation values, respectively, in a local neighborhood of ( x, y). The size of the neighborhood should be small enough to preserve local details, but at the same time, large enough to suppress noise. The value of k is used to adjust how much of the total image object boundary is taken as a part of the given object. In this research 15х15 neighborhood and k= - 0.2 gave well separated defect objects in the weld area. 6. FEATURE EXTRACTION In this article features describing number, size, location and availability of defected area is used to make distinction between true and faulty areas in the weld region. 6.1. Film defect feature During the development process of the film, sometimes weld area in the image is damaged by nail effect of the operator. This film defect is very similar to flaw patterns and may be mistaken instead of weld defect. As shown in figure 4 , the gray level of film defect is higher than gray level of real weld defect. So, in this feature the gray level of defected area is detected and if it is more than a defined threshold, the feature equals one. Otherwise the feature equals zero.

(4.a) (4.b) Fig.4. (a) Image with film defect(nail effect). (b) Image with weld defect,

8

6.2. Unavailability of defected area If there is no separated area known as defect , this feature equals to one, otherwise it equals to zero. 6.3. Average size of defects:

Average size of defects( nAi∑ ): the ratio between sum of defected areas (∑ iA ) to the number

of defects( n ) 6.4. Average distance of defects from center line:

Average distance of defects from center line ( nd i∑ ) : the ratio between sum of the distances of

each defect from the center of the weld bead (∑ id ) to the number of defects( n ).

6.5. Number of defects: Number of defects ( n ): number of separated areas in the weld bead known as defected area. 6.6. Zoning: Zoning is an important method for deriving low resolution structural features [13],[14]. In this study, zoning is adopted with the structure of the welded area. Welded region is extracted to a rectangular frame array. In normal zones height(H) and width(W) are selected to be the same, where W is always equal to the weld width. In proposed zoned block, height of the welded area is selected to be one third of W. Features mentioned in subsections 6.1 to 6.5 are extracted for these two types of zoned blocks. The first set is for normal zone(H=W) and the second one is for proposed zone(H=W/3). 7. CLASSIFICATION The task of pattern classification is to assign an input pattern represented by a feature vector, to one of the output specified classes. In complex systems with nonlinear relations between inputs and outputs, conventional approaches proposed for solving the problem of nonlinear pattern classification can be found in certain well-constrained environments, non is flexible to perform well outside its domain. Artificial neural networks (ANNs ) classifier can provide flexible alternatives and many applications could benefit from using them. ANNs are composed simple elements operating in parallel. These massively parallel systems with large number of interconnections may solve a variety of challenging classification problems. One of the more update lines of research is the classification of weld defects using ANNs and also there are many reports on this subject [1],[4],[11]. In this section two types of neural networks structure to construct the relationship between system inputs and outputs will be explained. The first ANNs is a feed forward multi layer perceptron with back-propagation learning algorithm. The second one is a new hybrid neuro-logic classifier structure proposed in this research. 7.1. Multi layer perceptron neural network architecture Multi layer perceptron (MLP) is the most common type of feed-forward ANNs. The ability of learning is a fundamental property in neural network architecture. During learning process connections of the weights are updated. Error-correction learning rule and back-propagation learning algorithm is used for updating purpose of this MLP structure. Proposed MLP is composed of many interconnected neurons that often called input, output and hidden layers as shown in figure 5. The neurons of input layer are used to receive input vector X and neurons of output layer are used to produce the corresponding output vector Y. Pattern classifier in this section is made up of three layers, with five neurons in input layer, and two neurons in output layer. The activation function f is commonly chosen to be sigmoid in order to resemble the two state output of biological neurons which originally inspired the networks. Output vector is defined equal to ( 1Y =1, 2Y =0) for no defect status and equal to

9

( 1Y =0, 2Y =1) for defected status. Neurons in the hidden layers, sum up values from input nodes after

weighting them with appropriate weights jiW and compute the output Yo as a function of summation.

In training process, the actual output vector oY generated by network may not equal to the desired

output vector dY . The back-propagation (BP) algorithm is the most commonly adopted MLP training

algorithm and it is the most widely applied neural network architecture. BP computation algorithm is as follows [12]

(5.a) (5.b) Fig.5. (a) Configuration of artificial neural networks(ANNs) with one input, one hidden and output layers using back-propagation learning algorithm. (b) Trained MLP neural network structure with three layers and five input neurons. 1. Initialize the weights( jiW ) to small random values.

2. Randomly choose an input pattern )(µX 3. Propagate the signal forward through the network.

4. Compute liδ in the output layer( l

oii YO = )

])[( loidi

li

li YYhf −′= µδ

where lih represents the net input to the i th unit in the l th layer, and f ′ is the derivative of the

activation function f . 5. Compute the deltas for the preceding layers by propagating the errors backwards:

∑ −=′= ++ .1,),1()( 11LLlforWhf l

jl

ijli

li δδ

6. Update weights using

1−=∆ lj

li

lji YW δη

7. Go to step 2 and repeat for the next pattern until the error in the output layer is below a specified threshold or a maximum number of iterations is reached. Therefore the BP algorithm starts with the output layer and iteratively computes the δ values for the neurons in all layers. It is common to add a momentum factor into a BP algorithm to increase its learning speed. The momentum factor determines how much the previous weights change influences the new weight change. The new equation for jiW∆ is shown as follows

ljiij

lji WXW ∆+=∆ + ηδη1

where η is the ‘momentum’ coefficient with value 10 ≤≤ µ . The results are presented in section 8.

10

7.2. New hybrid neuro-logic architecture In section 7.1 all five extracted features are fed directly into an ANNs with BP learning architecture (Fig.5a) then trained network is tested and evaluated. In this section a new hybrid neuro-logic structure is proposed. It is a combination of neural networks and binary logic structures. Only three features in subsections 6.3 , 6.4 and 6.5 is fed into three input neurons with BP learning algorithm. Other two features in subsections 6.1 and 6.2 having binary values are directly fed into a logical structure.

(6.a)

(6.b)

Fig.6. (a) Trained MLP neural network structure using back-propagation learning algorithm with three layers and three input neurons. (b) Proposed hybrid neuro-logic structure.

11

Training is done according to the structure shown in figure (6.a) but neuro-logic structure of figure (6-b) combined of neural networks and logical gates is used for testing and classification. Blocks named AND, OR and NOT are common logical gates. Blocks called T are binarizer. If the input to this block is more than a predefined threshold, the output is one, otherwise the output is zero. Selected threshold is 0.5 for these blocks. If one of the features 6.1 or 6.2 becomes active, means there is no defect, regardless of the results in the ANNs classifier, the output will be defect free (ie. 1Y =0, 2Y =1) otherwise the results of the neural network classifier will be dominant. 8. RESULTS 8.1. Data Data bank was selected from images of γ -ray radiography of welds in 61 ′′ gas pipelines. Digitizing method was described in section 3. Ninety images of welds were selected from the data bank and used to train MLP neural network structure. Two third of the selected data were defect free and the remaining one third were defected. Error tolerance for training data was set at the value of 0.01 with the default learning rate of 0.05. By varying the number of neurons in the hidden layer and following the training errors, the best performance was reached at forty neurons in the intermediate layer which is optimum number of neurons for the data used. Table1. Recognition results of two types of classifier for two sets of features including normal zone(H=W) and proposed zone(H=W/3). No film defect data used for training.

Data with no film defects Correct

Recognition (%) Rejection (%) Error (%)

BP structure (Fig.5)

85 13.4 1.6 Features from normal zone

(H=W) Neuro-logic structure (Fig.6)

88.3 10 1.7

BP structure (Fig.5) 90 10 0 Features from

proposed zone (H=W/3) Neuro-logic

structure (Fig.6) 93.3 6.7 0

Table2. Recognition results of two types of classifier for two sets of features including normal zone(H=W) and proposed zone(H=W/3). Film defect data used for training.

Data including film defects Correct

Recognition (%) Rejection (%) Error (%)

BP structure (Fig.5)

70 28.3 1.7 Features from normal zone

(H=W) Neuro-logic structure (Fig.6)

85 11.7 2.3

BP structure (Fig.5)

73.3 25 1.7 Features from proposed zone

(H=W/3) Neuro-logic structure (Fig.6) 88.3 10 1.7

8.2. Recognition Performance Recognition performance is shown in two tables. Both of the tables have the results of two types of classifier for two sets of features. The first classifier is ANNs with BP structure and the second one is the new hybrid neuro-logic structure proposed in this paper. One of the feature sets is extracted from

12

the normal zones(H=W) and the other one is obtained from proposed zone(H=W/3). Proposed neuro-logic structure has less input neurons and takes shorter time for training in comparison with BP structure with five input neurons. It has also better recognition performance according to the results shown in tables 1 and 2. Comparing the results from the zoning point of view indicates that in similar classifiers, weld separation efficiency increases for the features extracted from the proposed zone(H=W/3). Table 1 shows data with no film defect information used for training. In table 2 , seven film defect data are replaced in train set for training the classifiers. It is observed that directly using the film defect information in training data, reduces recognition efficiency. 8.3. Discussion Test results indicates that proposed neuro-logic structure gives better recognition performance and also speeds up the training process by separation between binary(two level) and numeric features, then training only numeric features and combining the whole results together by a logical structure. Although direct training of neural network structure with film defect information, reduces recognition efficiency and creates more confusion in weld separation stage, but this new neuro-logic structure have the ability to reject film defects by its logical structure with no need for film defect training. Also more meaningful results can be obtained from features extracted from proposed zone(H=W/3). Overall recognition rate and reliability can be increased by developing data samples and using a big training set. 9. CONCLUSION This paper has described a method for automatic recognition and separation between defected radiographic images of welds and correct ones. A new structure of neural network classifier in combination with binary logic is introduced. Experimental results show that this new hybrid neuro-logic structure gives better recognition performance, especially when ANNs are confused by film defect information during training stage. Neuro-logic structure has the ability to reject film defects by its logical part with no need for training the film defects. It is important because direct training of the film defect information to neural networks, reduces the recognition efficiency. For better recognition performance adaptive preprocessing and local binarization techniques are used for image enhancement and segmentation then features are extracted from proposed zoned blocks. 10. REFERENCES 1. da Silva, R.R., Caloba, L.P., Siqueira, M.H.S. and Rebello, J.M.A.(2004). Pattern Recognition of Weld Defects Detected by Radiographic Test. NDT & E International, 37:461-470. 2. Gonzalez, R.C. and Wintz, P.(1987). Digital Image Processing. Addison-Wesley, 2nd Edition. 3. Wanga, X. and Wong, B.S. (2004). Image Enhancement for Radiography Inspection. The Third international Conferenceon Experimental Mechanics, SPIE Proceedings, 5852. 4. Wang, G. and Liao, T.W.(2002). Automatic Identification of Different Types of Welding Defects in Radiographic Images. NDT & E International,35:519-528. 5. Shafeek, H.I., Gadelmawla, E.S., Abdel-Shafy, A.A. and Elewa, I.M. (2004). Assessment of Welding Defects in Pipeline Radiographs Using Computer Vision. NDT & E International,37:291-299.

6. Castleman, K.R.(1979). Digital Image Processing. Englewood Cliffs N.J., Prentice Hall, 90-94. 7. Wang, G. and Liao, T.W. (2002). Automatic Identification of Different Types of Welding Defects in Radiographic Images. NDT & E International, 35:519-528. 8. Liao, T.W. and Ni,J. (1999). An Automated Radiographic NDT System for Weld Inspection: Part I- Weld Extraction. NDT & E International, 29:157-162. 9. Trier, Ф.D. and Taxt, T. (1995). Evaluation of Binarization Methods for Document Images. IEEE Transactions on Pattern Analysis and Machine Intelligence,17:312-315. 10. Trier, Ф.D. and Jain, A.K. (1995). Goal Directed Evaluation of Binarization Methods. IEEE Transactions on Pattern Analysis and Machine Intelligence,17:1191-1201. 11. Juang, S.C., Tarng, Y.S. and Lii, H.R. (1998). A comparison between Back-propagation and Counter-propagation Networks in the Modeling of TIG Welding Process. J of Materials Processing Technology, 75:54-62.

13

12. Jain, A.K., and Mao, J. (1996) Artificial Neural Networks : A Tutorial. IEEE Computer Magazine, March 1996:31-44. 13. Trier, Ф.D. and Jain, A.K. (1996) Feature Extraction Methods for Character Recognition –A Survey. Pattern Recognition, 29:641-662. 14. Srikantan, G. et al. (1996) Gradient Based Contour Encoding for Character Recognition. Pattern Recognition, 29:1147-1160. 11. LIST OF TABLES Table1. Recognition results of two types of classifier for two sets of features including normal zone(H=W) and proposed zone(H=W/3). No film defect data used for training. Table2. Recognition results of two types of classifier for two sets of features including normal zone(H=W) and proposed zone(H=W/3). Film defect data used for training. 12. LIST OF FIGURES Fig.1. Digitized images with different resolution and 8 bits of gray level. (a)300 dpi. (b) 600 dpi. (c) 1600 dpi. (d) 2000 dpi. Fig.2. (a) 5Χ5 neighborhood pixels as a mask for median filtering in Eq.(1). (b) Original image (c) 5Χ5 median filtered image. Fig.3. (a) Original image. (b) Histogram of original image. (c) Adaptive histogram equalization(AHE), The contextual region shown is an m х m mask around a pixel at location (x,y). (d) Image after AHE (e) Histogram of image after AHE. Fig.4. (a) Image with film defect(nail effect). (b) Image with weld defect, Fig.5. (a) Configuration of artificial neural networks (ANNs) with one input, one hidden and output layers using back-propagation learning algorithm. (b) Trained MLP neural network structure with three layers and five input neurons. Fig.6. (a) Trained MLP neural network structure using back-propagation learning algorithm with three layers and three input neurons. (b) Proposed hybrid neuro-logic structure.