Medical Image Retrieval and Classification Based on Morphological Shape Feature

Fu Li-dong School of Information Science and Engineering,

Shenyang Ligong University, Shenyang 110159, China fulidong@sina.com

Zhang Yi-fei School of Information Science and Engineering,

Northeastern University, Shenyang 110004, China zhangyifei@ise.neu.edu.cn

Abstract—Medical Image Retrieval and Classification is very important in Computer-Aided Diagnosis. Feature extraction is one of the most important techniques in content based image retrieval and classification. How to extract low-level features which reflect high-level semantics of an image is crucial for medical image retrieval and classification. In allusion to this issue, there proposed a method using edge density histogram to extract shape feature of medical images in this paper. Then Euclidean distance and Support Vector Machine (SVM) are used for medical image retrieval and classification. Results of experimentation showed that the proposed algorithm has been applied to medical image retrieval with promising effect.

Keywords—image retrieval; shape feature; morphological; histogram；support vector machine

I. INTRODUCTION With the popularity of medical imaging equipment, a

growing number of medical images are applied to medical diagnosis. These images can provide information of various pathologies and information of identification for disease diagnosis. Medical image database has become to an important part of medical diagnosis and preventive medicine[1]. How to quickly and efficiently locate these images, how to retrieve and classify images, how to find the hidden patterns and knowledge in them, how to facilitate the doctors to make diagnosis of new diseases is a challenging medical problem today. In recent years, methods of Content-Based Image Retrieval(CBIR) are widely used in image retrieval instead of Text-Based Image Retrieval. This can not only overcome the issues of long time, large human consumption and the lack of integrity and objectivity caused by manual annotation, but also can import the abundant visual features of images to retrieval systems.

Research on medical image retrieval and classification is an important part of multimedia data mining. Image retrieval results depend largely on the feature extraction. However, due to the diversity of image feature extraction, the different evaluation methods and clinical requirements make the extraction of medical images information to an extremely difficult task. For different mining methods, the extraction of semantic features also has large differences. Manjunath did a comprehensive summary on the color and texture descriptor on semantics, extraction and storage[2]; WAN proposed a new extraction method of texture and edge descriptor, and on that basis, he integrated texture, edge and color histograms as image feature vector and achieved semantic classification[3]; Fung proposed a Newton method and extracted a small amount of features as classification features of images through processing the high-dimensional space data [4].

Edge as one of the basic features of images contains contour information of object in images. Therefore, edge features can not only express the image content but also use for object recognition. Although the Prewitt, Sobel and Canny descriptors can well separate the edge information from the background, but the edge image is generally complicated and can not constitute image contours, and those huge number of lines and curves are difficultly described by mathematical formula[5]. Rosin proposed an approximation method using shape to express the image edge, by using shapes like line, circle, oval, arc and polygon. But this method is very complex and time-consuming[6]. Although WAN described six kinds of edge type in the spatial distribution of images, he did not specify the detailed steps of type judgements[3]. Ren Ping-hong and others proposed an improved algorithm of edge histogram, but it ignored the inside details of objects and background [7].

In this paper, we proposed a new descriptor which extended edge histogram method of WAN, combined local features with global shape features, combined edge of whole image with edge density of sub-images, it is named as Edge Density Histogram Descriptor (EDHD). Firstly image edges were detected by a multi-scale morphological gradient algorithm. Then shape features were extracted from the obtained edge image and edge-density histogram was constructed. Lastly medical image retrieval and classification was executed according on Euclidean distance and support vector machine. This method combines the global and local features of images, achieves content-based medical image retrieval and classification well. Results of experimentation showed that this method was effective for medical image retrieval and classification.

II. EDGE EXTRACTION In digital image processing, usually combine

morphological gradient operator and threshold method to accomplish edge detection. If the gradient is bigger in the image somewhere, it indicates that the brightness of there changes in a larger scale and the edge maybe passes there [8]. Generally, these gradients will be given in the form of digital difference.

Three kinds of morphological gradient operators are defined as follows:

(1) Expansion operation of gradient operator ( , ) ( , ) ( , ) ( , )G x y f x y B x y f x y= ⊕ − (1)

(2) Corrosion operation of gradient operator ( , ) ( , ) ( , ) ( , )G x y f x y f x y B x y= − ○ (2)

(3) Composite operation using Expansion and Corrosion of gradient operator

2010 Third International Conference on Intelligent Networks and Intelligent Systems

978-0-7695-4249-2/10 $26.00 © 2010 IEEE

DOI 10.1109/ICINIS.2010.86

116

( , ) ( , ) ( , ) ( , ) ( , )G x y f x y B x y f x y B x y= ⊕ − ○ (3) Where ⊕ and ○ respectively denote Expansion and

Corrosion operation.

(a) (b) (c) (d)

Figure 1. Edge extracting results: (a) Morphological edge (b) Binary edge (c) Single-pixel Edge (d) Canny edge

As the morphological operation is a computation based on geometric features of the signal. The Corrosion and the open operation can suppress the peak noise, and Expansion and close operation can suppress the valley noise. So we can use a combination of the four operations to achieve the purpose to reduce noise. In addition different structuring elements can not only revise the edge details of the image from different scale, but also detect various geometric shapes from different levels. Therefore, in this paper we proposed a composite operation using Expansion and Corrosion of multi-scale and structuring elements to edge detection, the computational formula is:

( ) ( ) ,1 1i n i nG f B B f B B i n= ⊕ − • ⊕ ≤ ≤ − (4) Where 1( )i i if B f B B−= ⊕ 1( )i i if B f B B−• = • ○ Experiments showed when the structuring element is

selected by 3×3 and 5×5 window, the computing speed is fast, and n = 3 make delicate enough to enhance the edge. In this paper, we use 3×3 window as the structuring element which selected the form of tapered, cylindrical, and diagonal. Experiments had been made to prove that it can get the best results. Structuring element which we used were:

1 2 3

1 2 1 0 1 0 1 0 12 8 2 , 1 1 1 , 0 1 01 2 2 0 1 0 1 0 1

B B B⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥ ⎢ ⎥= = =⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦

The results of this edge extraction are showed in Fig. 1(a). Since the pixel gray level of the extracted edge image is less and edges are not prominent, in order to facilitate the feature extraction subsequent, in this paper we use the weighted average threshold to do the gray image preprocessing. See fig. 1(b). Furthermore, in order to effectively determine the edge type, using the invariableness rules of Euler number to gradually refine the binary edge image and finally get the single-pixel edge image, See fig. 1(c). Fig. 1 (d) shows the results of Canny edge descriptor. From these images the edge detection algorithm of multi-scale and structuring elements can be seen has good anti-noise performance and better contour extraction effect.

III. EXTRACTION OF SHAPE FEATURE In this paper, the edge features extracted include both

shape histogram and density histogram. To think over local information of images, we separate image to several sub-regions. By analysis medical images in image database we found that most of the medical images are 512×512 size. So images which don’t conform to this standard size are modified to the size of 512×512 by using the structural

scaling methods. Because images of the same class have the same aspect ratio, using the approach of scaling does not affect the search results. Each image is divided to 4×4 sub-regions in order to extract edge type and edge density.

A. Extraction of shape histogram Shape histogram is a kind of visual description for the

shape features of images. In this paper, based on the shape histogram of WAN we used six shape types to describe information of contour for every sub-region. They are empty, horizontal, 45o direction, vertical，135o direction and chaos, showed in Fig. 2(a).

(a)

(b)

Figure 2. Edge types and discriminance: (a) Edge types (b) Corresponding discriminance

Furthermore in this paper, sub-block of 8×8 size is selected as the basic unit to judge the edge type. By partitioning the sub-region to sub-block, each sub-region can be separated to Nall = 256 sub-blocks, each row has Ncol = 16 sub-blocks, and there is a total of Nrow = 16 lines.

Taking into account the complexity of image border, in this paper we use similar matching method to judge the six types. The type of sub-block will be extended description. See fig. 2(b), in every sub-block the number of pixels on the same position of the little circle on the sketch map are computed. Then determine the edge type of each sub-block

(0 ,0 )ij row colM i N j N≤ ≤ ≤ ≤ by using the threshold. Steps of this discriminance. (1) Establish four Boolean matrix M0, M45, M90, M135

correspond to the four edge types "Horizontal", "Vertical", "45o direction" and "135o direction"(See the sketch map , location of the circle is 1, the remaining position is 0), set threshold as Tblank and T;

(2) Compute the total number of edge pixels in the sub-block Mij, denoted as TotalPix;

(3) If TotalPix ≤ Tblank, then the sub-block is "Empty" type;

(4) Otherwise, do AND operation between matrix Mij and the four auxiliary matrixes, statistics the sum of edge pixels which match the sketch map of each edge types.

• If the sum of edge pixels which match the sketch map is larger or equal to T and the sum of edge pixels which not match is less than T, this sub-block belongs to the corresponding type;

• If the sum does not satisfy the four types mentioned above, this sub-block belongs to "Chaos" type.

Empty 0o 90

o 135

o Chaos45o

117

Using ( , )ET i j to express the edge type of sub-block on the position of row i and column j , where 0,1, , ,rowi N=

0,1, , colj N= . Then we can get the edge histogram of each sub-region by formula as following.

1 1( , )

( )

row colN N

i jsub

all

E i jH k

N= ==∑∑

(5)

Where1, ( , )

( , ) , 0,1, 50,

T i j kE i j k

others=⎧

= =⎨⎩

, k express

the edge type. Then, combining all edge histogram of sub-region, we

can get an edge histogram which has 6(types) × 16(sub-regions) = 96 handles for each image.

B. Extraction of density histogram Density histogram is a visual description of the

probability distribution of the edge in the regions. Compared with edge histogram, density histogram in this paper is more simple and intuitive. We use the ratio of edge pixels in sub-region and the total edge pixels of the whole image as the edge density of this sub-region. By calculating the edge density of these sub-regions we can get the density histogram of the whole image. The formula of density histogram is:

0 0

0 0

( , )( )

( , )

Subheight Subwidth

i jden height width

m n

p i jH s

image m n

= =

= =

=∑ ∑

∑ ∑ (6)

Where, ( , )p i j means the edge value of the pixel on the position of row i and column j , ( , )image m n express the edge value of the pixel on the position of row m and column n , subheight and subwidth are the height and width of sub-regions, height and width are the height and width of the whole image.

In this paper, firstly edge histogram is extracted and normalized, then combined with density histogram, so that we can get Edge Density Histogram of images. Each image can described by the feature data include 112 fields, and it can satisfy the needs of content-based medical image retrieval.

IV. RETRIEVAL AND SVM CLASSIFICATION

A. Image retrieval based on similarity measurement In this paper, the weighted Euclidean distance is used to

measure the similarity of images. Let 1 2( , , , )Q Q QNh h h and

1 2( , , , )I I INh h h as the normalized feature of image Q and I .

2

1( , ) ( )

NQ I

w i i ii

D Q I w h h=

= −∑ (7)

Where the value of iw is obtained by experiments. After doing repeated tests for the image retrieval system, the reasonable value of iw is:

0.6, 0 95(0.4,95 111(i

iw

i≤ ≤⎧= ⎨ < ≤⎩

Edge hi st ogr am）Densi t y hi st ogr am）

(8)

As normalized shape features are used, the weighted distance is less than the Euclidean distance.

B. Image classification based on support vector machine(SVM) SVM is a good tool for data classification. The basic idea

is that the data is mapped to a high dimensional space and find the hyperplane with the largest edge. Consider a binary classification problem which contains m samples.

Suppose eigenvector of training set is ,nkX R∈

1, 2, ,k m= . Each vector has a class mark { }1,1kY ∈ − . SVM is to solve a quadratic programming problem:

, , 1

1min2

mT

kw b k

w w C ξξ

=

⎛ ⎞+⎜ ⎟⎝ ⎠

∑ (9)

Limited to: ( ( ) ) 1 , 0, 1, , .T

k k k ky w x b k mφ ξ ξ+ ≥ − ≥ = (10) Data of the training set is mapped to a high dimensional

space through functionφ , C is the penalty parameter. For training object, its determinant function is

( ) sgn( ( ) )Tf x w x bφ= + (11) In practice, we need to select the appropriate kernel

function ( , ) ( ) ( )Tk x x x xφ φ′ ′= to determine a SVM. There are four basic types of SVM kernel functions: linear, polynomial, radial basis function and two layers neural network.

In this paper, we use the radial basis function as kernel function:

2( , ) exp( || || ), 0k x x x xγ γ′ ′= − − > (12) We use cross-validation method to choose the best

penalty parameter C and kernel parameterγ . Then use the best parameters C and γ on the entire test set to get the model of SVM .

V. EXPERIMENTS In our experiments we used BMP images in the database,

including lung X-ray, lung CT, CT/MRI images of brain and head. The total number is 958 images.

A. Retrieval We do experiments and record the response time of any

query command on different capacity of database. After a large number of experiments it showed that using features mentioned in this paper, the response time is 4 second when retrieval in database of 300 images. Along with growth of the database, the response time increases proportionally.

By setting the same similarity distance ( 0.2d ≤ ), we carried out the performance evaluation for various types of image retrieval on average precision and recall. See Table 1, in addition to the recall rate of lung-X images is lower, the rest of the images all have a recall rate about 90%. In precision, since the image of brain and head both have CT and MRI images, and these two kinds of images have a strong similarity on shape, so when the similarity is limited under a value their precision is lower, but their recall is higher.

118

TABLE I. AVERAGE RECALL AND PRECISION OF VARIOUS IMAGES

Category Recall Precision Lung-X 0.765 0.785 Brain 0.890 0.765 Head 0.905 0.523

Lung-Ct 0.970 0.980 In our experiment, we compared Edge Density

Histogram Descriptor of multi-scale and structuring element (MSS) with Canny edge descriptor and single structuring element (SS). Fig. 3 shows the compared result of these three kinds of image retrieval methods on brain MRI images. We can conclude that it has better results by using the method in this paper.

10 20 30 40 50 60 70 80 90 100

30

40

50

60

70

80

90

100

Prec

isio

n %

Recall %

MSS Canny SS

Figure 3. Performance estimate of image retrieval

B. Classfication We separated all the images to three groups. Each group

has two kinds of images. In each group, 180 images is selected from this two types of images as the training set, the remaining images as the test set. Using the radial basis function as kernel function, using cross-validation method to choose the best penalty parameter C and kernel parameter γ . Then use these parameters to do the training on the entire training set to get the SVM model, and classify in test set. The accuracy of classification is defined as the following formula:

right

all

NP

N= (13)

Where rightN is the number of images which are classified right, and allN is the total number of images.

TABLE II. CLASSIFICATION RESULT OF LUNG-X AND BRAIN

Descriptor C γ Lung-X Brain SS 22 2-3 85.24 84.38

Canny 23 2-5 91.25 91.07 MSS 26 2-2 94.88 92.15

TABLE III. CLASSIFICATION RESULT OF HEAD AND BRAIN

Descriptor C γ Head Brain SS 24 2-4 78.38 79.78

Canny 25 2-6 84.87 81.37 MSS 22 2-5 87.17 86.98

TABLE IV. CLASSIFICATION RESULT OF HEAD AND LUNG-CT

Descriptor C γ Head Lung-CT SS 24 2-6 88.75 82.24

Canny 23 2-3 92.67 92.75 MSS 25 2-4 94.59 95.78

The results of classification are showed in Table II to IV. From the results we can conclude that it has better result of classification than other methods by using MSS method in this paper.

VI. CONCLUSION In this paper, we proposed a method of medical image

retrieval and classification based on morphological edge detection. In this method, we extract the edge of images through a multi-scale and structuring elements method, then build edge density histogram to extract shape features, and use this feature to do the image retrieval and classification. Results of experimentation show that the shape features mentioned in this paper can fully describe the content of image, and improve the recall and precision rate and classification accuracy on medical image retrieval.

In the system implementation process, the features of images are stored in the image database uniformly. When users submit queries, the system retrieves all the images in database, and then returns the result. This makes the system slow down significantly when the image database gradually increasing. In order to improve the speed, while ensuring the quality of image features under the premise there is an urgent problem need to be solved that how to optimize the index structure of image features in database. And that is the major work which we will do in the future.

REFERENCES

[1] H. Muller, N. Michoux, D. Bandon and A. Geissbuhler, “A review of content-based image retrieval systems in medical applications-clinical benefits and future directions,” International Journal of Medical Informatiocits, vol. 73(1), pp. 1-23, 2004.

[2] B. S. Manjunath, J. R. Ohm, V. V. Vasudevan and A. Yamada. “Color and texture descriptors,” IEEE Transaction on Circuits and Systems for Video Technology, vol. 11(6), pp. 703-714, 2001.

[3] H. Wan and M. U. Chowdhury. “Image semantic classification by using SVM,” Journal of Software, vol. 14(11), pp. 1891-1899, 2003.

[4] G. Fung and O. L. Mangasarian, “A feature selection newton method for support vector machine classification,” Computation Optimization and Application, vol. 28(2), pp. 185-202, 2004.

[5] A. Pentland, R. W. Picard and S. Sclaroff, “Photobook content-based manipulation of image database,” Journal of Computer Vision, vol. 18(3), pp. 233-254, 1996.

[6] P. L. Rosin and G. A..West, “Nonparametric segmentation of curves into various representations,” IEEE Transactions on PAMI, vol. 17(12), pp. 1140-1152, 1995.

[7] Ren Ping-hong and Chen Chu, “Methods of image retrieval based on improved edge histogram,” Computer Technology and Development, vol. 17(8), pp. 183-186, 2007.

[8] John Canny, “A computational approach to edge detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 8(1), pp. 679-697, 1986.

119