Vertebral Body Segmentation of Spine MR Images Using Superpixels

Paulo D. Barbieri1, Glauco V. Pedrosa1, Agma J. M. Traina1 and Marcello H. Nogueira-Barbosa21Instituto de Ciencias Matematicas e de Computacao (ICMC/USP)

2 Departamento de Radiologia, Faculdade de Medicina de Ribeirao Preto1,2University of Sao Paulo (USP), Brazil

paulo.barbieri@usp.br, {gpedrosa,agma}@icmc.usp.br

AbstractThis paper presents a segmentation approachguided by the user for extracting the vertebral bodies ofspine from MRI. The proposed approach, called VBSeg, takesadvantage of superpixels to reduce the image complexity andthen making easy the detection of each vertebral body contour.Superpixels adapt themselves to the image structures, oncetheir formation law follows the homogeneity of the imageregions. However, for some diseases or abnormalities, theboundary of each superpixel does not t well in the vertebracontour. To avoid this drawback, we propose to use the Otsusmethod as a pos-segmentation step to divide the superpixelsinto smaller ones. The nal segmentation is obtained througha region growing approach using points manually selectedby the specialist. It can produce masks of the ve lumbarvertebrae with an average precision of 80% and recall of87%, when compared to the manual segmentation of a trainedspecialist. These values show that the VBSeg is a valuable assetto assist the medical specialist in the task of vertebral bodiessegmentation, with much less effort and time demand.

Keywords-image processing; segmentation; superpixel;medical image; spine

I. INTRODUCTIONMagnetic Resonance Imaging (MRI) is frequently used to

diagnose spinal diseases and abnormalities, such as vertebralfractures and deformities, slipped vertebra, bone marrowabnormalities, herniated or degenerated disc. However, thevertebral detection and segmentation from MRI is stilla challenging problem in this area. The segmentation isof crucial importance for creating a CAD (Computer-Aided Diagnosis) system, which allows the medicalspecialist to draw conclusions about the condition of thepathology and helping in evaluating treatment for patients.Many spine abnormalities or deformities such as scoliosis(curvature in anatomical left-right direction), vertebralfracture (crushed vertebra) and spondylolisthesis (misalignedvertebra) can be diagnosed from vertebral shapes, positionsand orientations, so an accurate vertebra segmentation is ofutmost importance.

In the last decade, spine image processing and analysiswere extensively studied and some methods were proposed[1]. Previously published works on segmentation of thehuman spinal cord have involved several classes ofalgorithms, such as contour and edge detection [2], seededregion growing [3] and B-spline active surface [4]. However,previous works generally require high computational effort

which can be time-intensive and not suited to an interactivescenario like a CAD.

In this paper, we propose an accurate and semi-automatic approach to assist the medical specialist inthe task of segmenting the vertebral bodies of spineimages. The proposed method, called VBSeg (Vertebral BodySegmentation), allows extracting only the vertebral bodiesof interest to the medical specialist. In this paper, we areespecially interested in segmenting the ve lumbar vertebrae(L1, L2, L3, L4 and L5) of spine MR images.

The proposed segmentation is formulated as a superpixelgrouping problem, based on the observation that vertebralcontours are often reasonably well approximated bysuperpixel boundaries. Then, we take advantage ofsuperpixels segmentation as a pre-processing step to reducethe image complexity into clusters of homogeneous pixels.This strategy facilitates the detection of the vertebraecontours while allowing delimitating the complete region ofeach lumbar vertebra.

We evaluated our method by comparing it with manualsegmentations (ground-truth) provided by radiologists. Theexperimental results show that the proposed approach is apowerful asset in helping the medical specialist by avoidingthe huge effort needed for a complete manual segmentationof the ve lumbar vertebrae.

The remainder of this paper is structured as follows:Section 2 gives the background needed to follow the workincluding the motivation. Section 3 explains the proposedsegmentation method. Section 4 presents the experimentalanalysis and Section 5 gives our conclusions.

II. MOTIVATION AND BACKGROUND

The human spine usually consists of 33 vertebrae: 7cervical (C1, C2, C3, C4, C5, C6, C7), 12 thoracic rangingfrom T1 to T12, 5 lumbar (L1, L2, L3, L4, L5), 5 sacralwelded forming the sacrum and 4 coccygeal, which mergeinto the coccyx.

The spinal cord can suffer many disorders or deformities.Osteoporosis, for example, is characterized by a decreasedbone mineral density being the most common metabolismdisorder, reducing bone resistance, making it more fragileand susceptible to fracture. This disorder is most commonin the elderly population and it affects especially women in

2015 IEEE 28th International Symposium on Computer-Based Medical Systems

2372-9198/15 $3.00 2015 IEEEDOI 10.1109/CBMS.2015.11

44

Figure 1. The scheme of the proposed approach to segment the vertebral bodies of spine MR images.

the menopause period. Moreover, cancer in the spine canbe primary or secondary. A secondary-cancer or metastasisis relatively common in the spine and it can predispose tomalignant vertebral compression fractures and put pressureon the spinal cord and nerves.

A crucial challenge for creating an effective CAD systemis the segmentation of vertebral bodies of the spine. Infact, nding relevant vertebrae in a large dataset of medicalimages are a valuable resource to aid the diagnosis of agiven pathology [5]. The disease or abnormality affectingthe vertebrae are generally characterized by details in thevertebral body, like the gray scale texture and/or its shape[6].

However, the fully automatic segmentation of vertebraeis a very challenging problem and, in the literature, wecannot nd a reasonable automatic approach for this task.A manual segmentation requires a huge effort and precioustime of radiologists. However, a semi-automatic method canbe much more viable and useful to the medical specialistif this task effectively reduces the effort needed comparedto a fully manual segmentation. A fast and accurate semi-automatic segmentation approach can be well integrated toa CAD system.

In this paper, we address the problem of vertebral bodiessegmentation using the idea of superpixels. In the computervision area, superpixels are becoming a very useful approachto reduce the image complexity. Superpixels representimages as a limited cluster of homogeneous pixels usingsome region-based segmentation [7]. They have become keybuilding blocks of many computer vision tasks, such asobject localization, depth estimation and segmentation.

Superpixels should adhere well to image boundaries, fastto compute, memory efcient, and simple to use [8]. Thereare many approaches to generate superpixels, each with itsown advantages and drawbacks that may be better suitedto a particular application. In the next section we present a

valuable technique that relies on the use of superpixels as apre-processing step to the vertebral body segmentation task.

III. PROPOSED APPROACH

In this paper, we propose an effective segmentationtechnique, called VBSeg (Vertebral Body Segmentation),developed to extract the vertebral bodies from Spine MRI.The VBSeg aims at segmenting the vertebrae L1, L2, L3, L4and L5 of the human spinal, receiving as input the SpinalMR Image and ve points manually selected by the user,such that each vertebra has one of these points in your area.Fig. 1 shows the scheme of the proposed approach, whichis summarized as follows:

1) First, given a Spine MR Image, the specialist selectsve points that correspond to the relevant lumbarvertebral bodies to be segmented;

2) Then, we segment the image using a superpixeltechnique to cluster homogeneous pixels whilereducing the complexity of the image;

3) To avoid missing important boundaries of the vertebralbodies, each superpixel is pos-segmented using anOtsu-based approach;

4) The nal segmentation is given by a region growingtechnique using the points selected by the specialist asseeds.

In the rst step, the aid of the specialist must be necessary,because it avoids detecting false vertebrae considered notrelevant in the diagnostic of the pathology. In the followingwe detail the other steps of the proposed approach.

A. Superpixel Segmentation

The idea of using superpixels is a valuable strategyemployed by the proposed segmentation approach. Thegoal of using superpixels is based on the observation thatvertebral contours are often reasonably well approximatedby superpixel boundaries. In fact, superpixels are useful

45

to reduce the image complexity and, thus, to facilitate thedetection of the contour of each vertebral body.

In this work, we use the SLIC (Simple Linear IterativeClustering) approach [9], which is a simple but effectivetechnique based on k-means to generate the superpixels. TheSLIC works in the CIELab color space and the algorithmreceives as input the quantity k of superpixels that willbe generated. So, in the beginning, the algorithm equallydistributes, in the image space, k initial centroids Ci =[li ai bi xi yi]

T , for i = 1...k, where [li ai bi]T

represents the color value of the pixel in the CIELab colorspace and [xi yi]T its coordinates in the image grid. Then,each image pixel j = [lj aj bj xj yj ]T is associatedto the closest similar centroid Ci according to the followingdistance function:

D(j, Ci) =

(dc10

)2+

(dsN/k

)2(1a)

where,

dc =

(lj li)2 + (aj ai)2 + (bj bi)2 (2a)ds =

(xj xi)2 + (yj yi)2 (2b)

N is the quantity of pixels in the image and k is the quantityof superpixels.

The equation dc gives the color distance of the pixels andds the space distance. These two distances are combinedin the nal distance D, proving a useful information aboutthe color and spatial distance. After all pixels have beenassociated to its closest centroid using equation D, theposition of the k initial centroids are uptaded and theprocedure begins again. In general, 10 iterations are enough.

In fact, the value of k is an important parameter providedto the SLIC approach. It refers to the number of desiredsuperpixels and it has a relevant impact in the quality ofthe nal segmentation. Increasing the value of k, moresuperpixels will be generated and more ne details of theimage are detected, such as illustrated in Fig. 2. So, thequantity of superpixels should be associated to the detailsthat is seek inside the image structures.

B. Thresholding by Otsus method

However, using only the superpixel segmentation is notsufcient to get a high quality segmentation of the vertebralbodies. The main reason is due to the fact that somepathologies affect the color of the vertebral bodies in theMRI. Thus, in some cases the superpixel boundaries do nott well to the vertebral body, such as illustrated by the twosamples in Fig. 3: the bottom images show the missingboundaries (in red) where the superpixel was not able tosegment perfectly the vertebrae.

(a)

(b)

(c)

Figure 2. Obtained superpixel segmentation using different values of k:(a) k = 50 (b) k = 100 (c) k = 2500.

To improve the quality of the nal segmentation ofthe vertebral bodies, a post-processing segmentation isperformed to divide the superpixels into smaller ones(if necessary). In this work, to make this division, wetake advantage of the Otsus method [10], [11], whichis an effective approach to divide a region into smallersignicant ones. The Otsus method exhaustively searchesfor a threshold T that minimizes the intra-region variance(the variance within the region), dened as a weighted

46

Figure 3. Samples where the simple superpixel segmentation was notable to effectively detected the vertebrae contours. The red lines show themissing contours.

sum of variances of the two (or more) regions. In theproposed thresholding method, the Otsus method is appliedto each superpixel, and then each disconnected regions areconsidered as new superpixels.

Fig. 4 shows the result after applying the proposedthresholding: in left-image the superpixels were not able todetect important vertebral contours, then applying the Otsusmethod (right-image) we can see that the boundaries t verywell to the vertebra contour.

C. Region Growing

The last step is to segment the vertebral bodies. For this,we carried out a region growing based on the color proximityusing the values of the superpixels in place of pixels values.

Figure 4. Obtained result after dividing the superpixels by the thresholdingmethod. The red lines in left-image show the missing contour.

Denition 1: In this work, we consider the value of asuperpixel as the average of the color values of all the pixelsthat compose it.

The region growing is performed as follows: we use theve points selected by the specialist as seeds, so in thebeginning each vertebra is composed only by the superpixelbelonging to its seed point. Next, this seed-superpixel iscompared to its neighbors: if the difference between theirvalues is lower than a threshold T , then the neighbor-superpixel is assigned to the vertebra and inserted into aqueue of superpixels to be analyzed. After comparing allthe neighbors, we get a superpixel from the queue and wecompare it with its neighbor using the same procedure donewith the seed-superpixel. These steps are repeated until thequeue is empty. To avoid an excessive growth, the numberof superpixels inserts in the queue can be limited.

47

IV. EXPERIMENTAL RESULTS

In this section, experimental results arepresented and discussed. We have performed a comparativeevaluation between the proposed approach and the manualsegmentation of vertebral bodies made by radiologists.

A. Ground-Truth Dataset

Our experimental dataset is composed of 39 spine MRimages T1 (T1-weighted images in the sagittal plane)provide by the Clinical Hospital with our university. In eachimage, the vertebrae L1, L2, L3, L4 and L5 were manuallysegmented by radiologists experts in the eld, giving a totalof 195 segmented vertebrae. Among these 195 vertebrae, 61are composed of fractures and the remaining 134 are normalvertebrae. Among the 61 fractured vertebrae, 47 are benignfractures due to osteoporosis, and 14 of them are malignantfractures due to cancer.

B. Evaluation Metrics

To analyze the results, we used the following metrics toassess the effectiveness of the proposed approach comparedto the manual segmentation:

Precision: is the percentage of pixels correctly classiedas belonging to the spine;

Recall: is the percentage of the relevant pixels that wereclassied as belonging to the spine;

Jaccard index: is calculated by the amount of pixelsthat two images have in common that is the ratio ofthe intersection of the two images by the union betweenthem.

These metrics are complementary and together theygive a robust information about the capacity of theproposed segmentation approach in classifying an MRI pixelbelonging to a vertebral body or not.

C. Obtained Results

Table I shows the average values and the StandardDeviation (SD) achieved by the VBSeg compared to themanual segmentation. We have performed an extensiveanalysis using different quantities of superpixels. In fact thequantity of superpixels should be associated to the dimensionof the granularity that is seek inside the structures, whichaffects quality of the nal segmentation.

Considering a segmentation with 2,500 superpixels, theproposed method achieved an average Precision of 80%, itmeans that the VBSeg did get the majority of the relevantinformation of the spine image. Moreover, it achievedan average Recall of almost 87%, which shows that theVBSeg was able to segment almost all relevant pixelsbelonging to the vertebral bodies. The Jaccard index reachedalmost 71%, showing the ability of the proposed methodto segment the relevant information with the minimumirrelevant information.

Table IVALUES ACHIEVED BY THE PROPOSED METHOD COMPARED TO

MANUAL SEGMENTATION.

Metric

Quantity of Superpixels676 1444 2500

Average SD Average SD Average SDJaccard 0.573 0.14 0.700 0.12 0.709 0.11

Precision 0.660 0.18 0.796 0.13 0.801 0.14Recall 0.834 0.10 0.855 0.08 0.869 0.06

Figure 5. Some segmentation results obtained by the proposed method(right) compared with manual segmentation (left).

However these values represent the average valuescomputed considering all ground-truth. If we analyzeindividually the results, for normal cases the VBSeg wasable to segment almost 100% of the vertebral bodies. Theworst segmentation result achieved 66% of precision. Fig 5illustrates some cases.

We also performed an analysis separated for eachpathology. Table II shows the values achieved by theVBSeg for vertebral bodies with malignant and benignfractures. The proposed method achieved better precision in

48

segmenting vertebral bodies with benign fractures. This isbased on fact that the malignant fractures affect the vertebracolor in the MRI. Malignant fractures are characterized bydark regions in the vertebra, so it is more difcult to segmentthem, because when applying the growing region, the colorof the malignant vertebra can be similar to the color of thebackground, affecting the nal segmentation.

Table IIVALUES ACHIEVED BY THE PROPOSED METHOD SEPARATED BY

PATHOLOGIES.

MetricFractures

Malignant BenignJaccard 0.618 0.745

Precision 0.679 0.848Recall 0.881 0.864

V. CONCLUSION

In this paper, we introduced a new segmentation approachto extract the ve lumbar (L1, L2, L3, L4 and L5)spine vertebrae from MRI. The proposed method, calledVBSeg (Vertebral Body Segmentation), takes advantageof superpixel segmentation as a pre-segmentation step tofacilitate the contour detection of each vertebra. However,using only the superpixel segmentation is not sufcient toeffectively extract the vertebrae contours, then we proposeto apply a pos-segmentation step using the Otsus methodto divide each superpixel into smaller superpixels. Thisprocedure has been shown very useful to segment importantcontour information of the vertebral bodies, specially inmalignant fractures. The nal segmentation is obtainedthrough a region growing approach that uses as seeds vepoints manually selected by the user specialist.

By experimental tests, the proposed segmentation hasdemonstrated being a valuable and fast asset for vertebralbodies segmentation. An important aspect to be consideredis the fact that a medical specialist needs a huge effort tosegment manually the vertebral bodies, and this procedurecan approximately take 15 minutes. The VBSeg allows toreduce this effort to the task of selecting only ve points,which can take only a few seconds. Moreover, this selectioncan be done by a technician or someone with a lower level oftraining, avoiding the need of precious time of a specialist.

A comparative experiment was performed analyzing theproposed segmentation with images manually segmentedby expert radiologists. The results showed that the VBSeghas an average Precision accuracy of 80% while having aRecall of almost 87%. These values demonstrate that VBSegapproach can contribute to clinical practices in helpingmedical specialists in the task of manual segmentationof vertebral bodies in an interactive environment, such asthe ones in CAD, allowing drawing conclusions about thepresence or absence of bone marrow abnormalities in thevertebral body.

ACKNOWLEDGMENT

This work is supported, in part, by FAPESP, CAPES,SticAMSUD, the RESCUER project, funded by theEuropean Commission (Grant 614154) and by theBrazilian National Council for Scientic and TechnologicalDevelopment CNPq/MCTI.

REFERENCES

[1] A. Suzani, A. Rasoulian, S. Fels, R. N. Rohling, andP. Abolmaesumi, Semi-automatic segmentation of vertebralbodies in volumetric mr images using a statistical shape+posemodel, Proc. SPIE 9036, Medical Imaging, vol. 9036, pp. 16, 2014.

[2] T. C. Mann RS, Constantinescu CS, Upper cervical spinalcord cross-sectional area in relapsing remitting multiplesclerosis: application of a new technique for measuring cross-sectional area on magnetic resonance images, J Magn ResonImaging, vol. 26, pp. 6167, 2007.

[3] C. M. Schmit BD, Quantication of morphological changesin the spinal cord in chronic spinal cord injury using magneticresonance imaging, IEEE Eng Med Biol Soc, vol. 6, pp.44254453, 2004.

[4] P. G. B. G. M. D. A. S. Coulon O, Hickman SJ,Quantication of spinal cord atrophy from magneticresonance images via a b-spline active surface model. MagnReson Med, vol. 47, pp. 11761185, 2002.

[5] Z. Xue, L. R. Long, S. Antani, and G. R. Thoma, SpineX-ray image retrieval using partial vertebral boundaries, inIEEE Symposium on Computer-Based Medical Systems, 2011,pp. 16.

[6] S. Antani, R. Long, and G.Thoma, Bridging the gap:Enabling cbir in medical applications, in Proceedings of 21stIEEE International Symposium on Computer-Based MedicalSystems, 2008. CBMS 08., June 2008, pp. 46.

[7] X. Ren and J. Malik, Learning a classication model forsegmentation, in Computer Vision, 2003. Proceedings. NinthIEEE International Conference on, 2003, pp. 1017 vol.1.

[8] R. Achanta, Finding Objects of Interest in Images usingSaliency and Superpixels, Ph.D. dissertation, IC, Lausanne,2011.

[9] R. Achanta, A. Shaji, K. Smith, and A. Lucchi, SLICsuperpixels compared to state-of-the-art superpixel methods,IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 34, no. 11, pp. 22742281, 2012.

[10] N. Otsu, A Threshold Selection Method from Gray-levelHistograms, IEEE Transactions on Systems, Man andCybernetics, vol. 9, no. 1, pp. 6266, Jan. 1979. [Online].Available: http://dx.doi.org/10.1109/tsmc.1979.4310076

[11] R. C. Gonzalez and R. E. Woods, Digital Image Processing(3rd Edition). Upper Saddle River, NJ, USA: Prentice-Hall,Inc., 2006.

49