10
THE INTERNATIONAL JOURNAL OF MEDICAL ROBOTICS AND COMPUTER ASSISTED SURGERY Int J Med Robotics Comput Assist Surg 2009; 5: 213–222. ORIGINAL ARTICLE Published online 2 April 2009 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/rcs.253 Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study Xiao Dong Guoyan Zheng* ARTORG Centre–ISTB, University of Bern, Stauffacherstrasse 78, CH-3014 Bern, Switzerland *Correspondence to: Guoyan Zheng, ARTORG Centre–ISTB, University of Bern, Stauffacherstrasse 78, CH-3014 Bern, Switzerland. E-mail: [email protected] Accepted: 18 February 2009 Abstract Background Accurate extraction of bone contours from two-dimensional (2D) projective X-ray images is an important component for computer-assisted diagnosis, planning or three-dimensional (3D) reconstruction. Methods We propose a 3D statistical model-based, fully automatic segmentation framework for extracting the proximal femur contours from calibrated X-ray images. The automatic initialization is an estimation of a Bayesian network algorithm to fit a multiple-component geometrical model to the X-ray data. The contour extraction is accomplished by a non-rigid 2D/3D registration between the statistical model and the X-ray images, in which bone contours are extracted by a graphical model-based Bayesian inference. Results The contour extraction algorithm was verified on both cadaver and clinical datasets, visually and quantitatively. Compared to the ‘gold standard’, a mean error of 1.6 mm was observed when the automatically extracted contours were used to reconstruct a patient-specific surface model. Conclusions Our statistical model-based bone contour extraction approach holds the potential to facilitate the application of 2D/3D reconstruction in surgical navigation. Copyright 2009 John Wiley & Sons, Ltd. Keywords statistical models; segmentation; X-ray fluoroscopy; Bayesian network Introduction Two-dimensional (2D) projective X-ray images are still playing a crucial role in diagnosis and surgery. Accurate extraction of bone contours from X-ray images is an important component for computer-assisted diagnosis (1–5), planning (6–8) or three-dimensional (3D) reconstruction (9–12). 2D projective X-ray images can vary in brightness and contrast as well as in the imaged region of the anatomy. Therefore, conventional segmentation techniques (1,7,8) can not offer a satisfactory solution and model-based segmentation is usually implemented to obtain robust and accurate results (3–5,9,13–15). In (3–5,14,16,17), 2D statistical models, an active shape model (ASM) or an active appearance model (AAM) are constructed from a training image set under the assumption that the images are taken from a cer- tain view direction. 2D statistical models can encode both the shape and Copyright 2009 John Wiley & Sons, Ltd.

Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

Embed Size (px)

Citation preview

Page 1: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

THE INTERNATIONAL JOURNAL OF MEDICAL ROBOTICS AND COMPUTER ASSISTED SURGERYInt J Med Robotics Comput Assist Surg 2009; 5: 213–222. ORIGINAL ARTICLEPublished online 2 April 2009 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/rcs.253

Automatic extraction of proximal femur contoursfrom calibrated X-ray images using 3D statisticalmodels: an in vitro study

Xiao DongGuoyan Zheng*

ARTORG Centre–ISTB, University ofBern, Stauffacherstrasse 78, CH-3014Bern, Switzerland

*Correspondence to: Guoyan Zheng,ARTORG Centre–ISTB, University ofBern, Stauffacherstrasse 78,CH-3014 Bern, Switzerland.E-mail: [email protected]

Accepted: 18 February 2009

Abstract

Background Accurate extraction of bone contours from two-dimensional(2D) projective X-ray images is an important component for computer-assisteddiagnosis, planning or three-dimensional (3D) reconstruction.

Methods We propose a 3D statistical model-based, fully automaticsegmentation framework for extracting the proximal femur contours fromcalibrated X-ray images. The automatic initialization is an estimation of aBayesian network algorithm to fit a multiple-component geometrical modelto the X-ray data. The contour extraction is accomplished by a non-rigid2D/3D registration between the statistical model and the X-ray images, inwhich bone contours are extracted by a graphical model-based Bayesianinference.

Results The contour extraction algorithm was verified on both cadaver andclinical datasets, visually and quantitatively. Compared to the ‘gold standard’,a mean error of 1.6 mm was observed when the automatically extractedcontours were used to reconstruct a patient-specific surface model.

Conclusions Our statistical model-based bone contour extraction approachholds the potential to facilitate the application of 2D/3D reconstruction insurgical navigation. Copyright 2009 John Wiley & Sons, Ltd.

Keywords statistical models; segmentation; X-ray fluoroscopy; Bayesiannetwork

Introduction

Two-dimensional (2D) projective X-ray images are still playing a crucial rolein diagnosis and surgery. Accurate extraction of bone contours from X-rayimages is an important component for computer-assisted diagnosis (1–5),planning (6–8) or three-dimensional (3D) reconstruction (9–12).

2D projective X-ray images can vary in brightness and contrast as well asin the imaged region of the anatomy. Therefore, conventional segmentationtechniques (1,7,8) can not offer a satisfactory solution and model-basedsegmentation is usually implemented to obtain robust and accurate results(3–5,9,13–15).

In (3–5,14,16,17), 2D statistical models, an active shape model (ASM)or an active appearance model (AAM) are constructed from a trainingimage set under the assumption that the images are taken from a cer-tain view direction. 2D statistical models can encode both the shape and

Copyright 2009 John Wiley & Sons, Ltd.

Page 2: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

214 X. Dong and G. Zheng

texture information learnt from training dataset, whichis helpful in improving robustness and accuracy innoisy images. Due to the limited convergence region,the 2D statistical model requires proper initialization.Fully automatic initialization can be accomplished bythe generalized Hough transformation (14), neural nets(15) or evolutionary algorithms (16,17). But both theinitialization and the segmentation performance rely onwhether the view direction assumption can be fulfilled.

3D statistical models are also used for 2D segmentationand 3D reconstruction from calibrated 2D X-ray images(9–12). One of the main advantages is that it can be usedfor segmenting an image taken from an arbitrary viewdirection. The initialization of the 3D model is usuallymanually defined (9–11). Due to the dense mesh of the 3Dstatistical model, fully automated solutions based on anevolutionary algorithm is computationally expensive (19).

Bayesian network-based approaches (20–22) havebeen used to identify or track objects. The Bayesiannetwork embeds the object information in a graphicalmodel, where the constraints among subparts of theobject are represented as potentials among nodes and thelocal image information corresponding to each subpart asobservation of the node. Bayesian network is also exploitedto find deformable shapes (23,24). Belief propagation isusually used to perform an approximate inference on thegraph to find a maximum a posteriori (MAP) estimate.

In this paper we propose an automated methodfor extracting the proximal femur bone contour fromcalibrated X-ray images. The clinical motivation for ourwork is to reconstruct a patient-specific 3D surface modelof the proximal femur from the extracted contours forcomputer-assisted, hip-surfacing surgery (25). In thispaper, we focus on the automated contour extraction.For this purpose, we developed a 3D statistical model-based fully automatic segmentation framework. Theautomatic initialization is accomplished by first fittinga simplified multiple-component geometrical model ofthe proximal femur to the X-ray images and then aligningthe 3D statistical model to fit the geometrical model.The statistical 3D model-based contour extraction is thensolved by a joint segmentation/non-rigid registrationprocedure, which iteratively updates the extractedcontours and the instanced 3D model to fit the X-rayimages. We use the projected silhouette of the 3Dstatistical model on each X-ray image as a 2D templateand accomplish the template-based contour extraction bya Bayesian inference on a graphical model. The update ofthe 3D model is achieved by a 2D/3D registration to fitan instanced 3D model with the extracted contours.

Methods

Image acquisition

In our study, calibrated X-ray images from a C-arm oran X-ray machine were used. When the intra-operativelyacquired C-arm images were used, due to the limited

imaging volume of a C-arm, we asked for four imagesfor the proximal femur from different view directions,of which two images focused on the proximal femoralhead and the other two focused on the femoral shaft. Thecalibrated X-ray image set was represented by I.

3D statistical model of the proximalfemur

A principle component analysis (PCA)-based 3D statisticalmodel M, with 4098 vertices of the proximal femur, wasconstructed from a collection of 13 sets of CT data ofthe proximal femur, as shown in Figure 1A. An instancegenerated from the statistical model with the parameterset Q = {α, β0, β1, · · · , β11} can be described as:

M : S(Q) = α(S0 +11∑

i=0

βiλ1/2i Pi) (1)

where S0 is the mean model, α is the scaling factor, and λiand Pi are the ith eigenvalue and the correspondent eigen-vector of the correlation matrix of the training dataset.

Automated initialization of the 3Dstatistical model

To find the initial rigid transformation T0 and parameterset Q0 to align the model instance S(Q0) with theobserved X-ray images, we modelled the proximal femurby a multiple component geometrical model consistingof three components, head, neck and shaft, which aredescribed by a sphere, a truncated cone and a cylinder,with parameter set Xgeo = {XH, XN, XS}, respectively, asshown in Figure 1B.

A graphical model was then constructed for thegeometrical model, as shown in Figure 1C. The constraintsamong components were encoded in the conditionaldistributions among nodes (20,22). π(XS), π(XN) andπ(XH) were the prior information for the shaft, neck andhead. The conditional distributions p(XN|XS), p(XH|XN)

were set so that the geometrical model could represent ameaningful anatomical structure of the proximal femur.A combination of particle filter and probability logicsampling, which can also be regarded as an estimation ofBayesian network algorithm (EBNA), was implemented[see details in (28)] to find an instance of the geometricalmodel X0

geo which fitted the X-ray images, as shown inFigure 1D.

From the mean shape of the 3D statistical model S0,the model vertices could be classified into three regions,femoral head, neck and shaft. The femoral head centreand radius and axes of the femoral neck and shaft couldbe determined in the model coordinate space by a 3Dsphere fitting to the femoral head region and cylinderfitting to the femoral neck and shaft regions. From theseextracted landmarks from the mean shape, a referencecoordinate system Cstat could be established, whose origin

Copyright 2009 John Wiley & Sons, Ltd. Int J Med Robotics Comput Assist Surg 2009; 5: 213–222.DOI: 10.1002/rcs

Page 3: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

Automatic extraction of proximal femur contours from calibrated X-ray images 215

Figure 1. Automatic initialization of the 3D statistical model. (A) PCA-based 3D statistical model. (B) A simplified multiplecomponent geometrical model representation of the proximal femur. (C) Graphical model for the multiple component geometricalmodel. (D) Fitting the simplified geometrical 3D model with X-ray images. (E) Fitting the statistical model with the estimatedgeometrical model

was located at the fitted femoral head centre, its x–y planeparallel to the plane constructed by the fitted femoral neckand shaft axes and its x axis parallel to the femoral neckaxis. Similarly, another coordinate system Cgeo could alsobe constructed, using the landmarks obtained from thegeometrical model X0

geo. The initial rigid transformationT0 and parameter set Q0 = {α0, 0, · · · , 0} could then becomputed to fit the statistical model (the scaled meanshape) to the geometrical model, as shown in Figure 1E.T0 was computed as the rigid transformation betweenthe coordinate systems Cstat and Cgeo. For both the meanshape of the statistical model and the fitted geometricalmodel X0

geo, we took two parameters, the femoral headcentre R and the distance between the head centre andthe femoral shaft axis D, as their scale indicators. Theinitial scale α0 was then determined by the mean ratio ofthese two parameters as:

(R0geo/Rstat + D0

geo/Dstat)/2.

3D statistical model-based contourextraction

After the initialization of the statistical model, the contourextraction was accomplished by joint registration andsegmentation, as summarized in Table 1.

Table 1. 3D statistical model based contour segmentation

1. Simulated X-ray and silhouette extractionGiven the current instanced statistical model M : S(Qn) and thetransformation Tn, project the aligned statistical model on each of the KX-ray image planes, using the projection geometry of each X-ray image.From the simulated X-ray images, the silhouettes {Ck,n

model}k=0,···,K−1 areextracted (1)2. 2D template-based segmentationOn each X-ray image, taking the correspondent silhouette of theprojected statistical model {Ck,n

model} as a template, a Bayesian

network-based shape matching is implemented to search for {Ck,nimage}

3. Non-rigid 2D/3D registrationA 2D/3D non-rigid registration is carried out to fit the extracted bonecontours {Ck,n

image}k=0,···,K−1 and the statistical model M, which results inan updated model instance M : S(Qn+1) and rigid transformation Tn+1

4. Repeat steps 1–3 a certain number of times (e.g. 2) or until theprocedure converges

2D template-based segmentation usingbelief propagation

From the silhouette of the projected 3D statistical model,we sampled M points (nodes) tracing along the contour asthe shape prior. Each point was described by a parameterset qi = {xi, gi, flagi}, i = 1, · · · M, where: xi = (xi, yi) wasthe position of the ith point in the image coordinate

Copyright 2009 John Wiley & Sons, Ltd. Int J Med Robotics Comput Assist Surg 2009; 5: 213–222.DOI: 10.1002/rcs

Page 4: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

216 X. Dong and G. Zheng

system; gi = (gxi , gyi) was set as the normal vectorof the template curve at position xi; flagi = 1 if thecurrent node belonged to the femur head projectionsilhouette; and flagi = 0 otherwise. The configuration ofour model could then be written as Qmodel = {qi}i=1,···M.The configuration of a candidate contour could bewritten as Qcand = {q′

i}i=1,···M, where q′i = {x′

i, g′i}, with

x′i representing the position of the ith candidate in the

image coordinate system and g′i being the gradient vector

of the X-ray image at position x′i.

We then established a partially connected graph withM vertices as:

G(V, E), V = {vi}i=1,···,M, E = {eij}i,j=1,···,M

where eij = 1 for (a) (|i − j| < NShaft) ∩ (i �= j) ∩ (flagi =0) ∩ (flagj = 0), (b) (flagi = 0) ∩ (flagj = 1), and (c) (|i −j| < NHead) ∩ (flagi = 1) ∩ (flagj = 1), as shown in Figure2A. NHead and NShaft determined the number of connectedneighbours for the head nodes and non-head nodes,respectively. Larger NHead and NShaft will keep the rigidityof the shape but will fail to track deviation from thetemplate. The reason that all the head nodes wereconnected with the non-head nodes was that we neededthe non-head nodes (which are supposed to be relativelyeasier to locate than the head nodes) to guide thelocalization of the head nodes. The corresponding factorgraph is shown in Figure 2B.

Given the template Qmodel = {qi}i=1,···M, the jointprobability distribution of the factor graph with ancandidate configuration Qcand = {q′

i}i=1,···M was then givenby:

p(Qcand) = 1Z

i

ψi(q′i)

eij=1

ψij(q′i, q′

j ) (2)

where ψi(q′i) = dot(gi, g′

i), which means to penalizecandidates with weak gradient amplitude and inconsistentgradient direction with the model, and:

ψij(q′i, q′

j ) = e−(µ

(x′i − x′

j ) · (xi − xj)

(||x′i − x′

j ||)(||xi − xj||)

+υ||||x′

i − x′j || − ||xi − xj||||

||xi − xj|| )

which was set so that the global shape of the modelwould be kept by penalizing the deviation of the angle

and distance between vertices from our model. µ and υ

are two weighting parameters to balance the two itemsin ψij.

Under these definitions, a bone contour that keeps theglobal shape of our model and at the same time locatesitself to the strong edge positions can be obtained by amaximal likelihood (ML) estimation as:

C∗image = max

Qcand={q′i}(∏

i

ψi(q′i)

eij=1

ψij(q′i, q′

j )) (3)

In our approach, the candidate positions for eachnode of the bone contour were sampled along thenormal direction of the model and standard loopy beliefpropagation (23) was used to approximate the MLestimation, as shown in Figure 3 (top row).

D.2 2D/3D non-rigid registration

Our statistical model could then be fitted to theextracted bone contours {Ck,n

image} using a 2D/3D non-rigid registration procedure (10). For each point Pl onthe extracted bone contour, the correspondence betweenits back-projection line BP(Pl) and a vertex vcorr(Pl)

on the current instantiated model M : S(Qn) could beestablished, using the current rigid transformation Tn.Projecting vcorr(Pl) onto BP(Pl) generates a correspondent3D point pair (vcorr(Pl), Proj(vcorr(Pl), BP(Pl))). A rigidtransformation Tn+1

update could be calculated to align thecurrent statistical model M : S(Qn) to the extractedcontours by minimizing the distances between all 3D pointpairs. The rigid transformation could then be updatedas Tn+1 = Tn+1

updateTn. The residual error between those

correspondent point pairs could be further reduced bygenerating a new instance M : S(Qn+1), using a statisticallyregularized instantiation (18). An example of the non-rigid registration is shown in Figure 3 (bottom row).

Experimental Results

We designed and conducted three experiments, asdescribed below, to validate the accuracy and robustnessof the present approach. The first two experimentswere performed on three clinical datasets, while thelast experiment was conducted on five cadaver datasets.

Figure 2. Bayesian network for the template based contour extraction. (A) Graphical model for the contour extraction, where thehead nodes are represented by filled dots and non-head nodes are shown as circles. (B) Factor graph for the graphical model forthe contour extraction

Copyright 2009 John Wiley & Sons, Ltd. Int J Med Robotics Comput Assist Surg 2009; 5: 213–222.DOI: 10.1002/rcs

Page 5: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

Automatic extraction of proximal femur contours from calibrated X-ray images 217

Figure 3. An example of 3D statistical model based contour extraction. Top row, 2D template-based contour extraction using beliefpropagation, where circles show the projected silhouettes and dots show the extracted contours. Bottom row, 2D/3D non-rigidregistration to fit the 3D statistical model to the extracted bone contours

Each clinical dataset contained only four calibrated C-armimages of the proximal femur, acquired intra-operativelyaccording to the protocol defined above. These areanonymized when they were provided to us indirectlyfrom a hospital through our collaborator (BrainLAB AG,Munich, Germany). No information about the model andthe manufacturer of the C-arm machine was available tous. In contrast, each cadaver dataset had two calibratedX-ray radiographs, which were acquired with a standardX-ray tube and generator (Philips Rotalix/Optimus,Philips Medical Systems, Eindhoven, The Netherlands)and then wer calibrated using a custom-made calibrationcage (26), and additionally, a ground truth surfacemodel of the associated bone that was established usingeither a hand-held laser-scan reconstruction method(T-SCAN, Steinbichler, Neubeuern, Germany) or a CT-scan reconstruction method (Amira, TGS Europe, Paris,France). For details about how the ground truth wasestablished, we refer to our previous work (26,27).

In the first experiment, we ran our algorithm on thethree clinical datasets with parameters set as M = 35,NHead = 4, NShaft = 3, µ = 2, υ = 1. The results are shownin Figure 4 (top, middle and bottom rows, respectively).Visually we were able to observe that our algorithm couldautomatically extract the overall contours of the proximalfemur from each image, although in certain local regionsit failed to find the correct contour locations.

In the second experiment, we performed a 2D/3Dsurface model reconstruction, using the contours auto-matically segmented from each dataset by the presentapproach. The 2D/3D surface model reconstructionscheme that we used here was a further improvementof the approach that we introduced in (10), whichcombined a statistical instantiation and a regularized

shape deformation with an iterative image-to-modelcorrespondence-establishing algorithm. Due to the lackof ground truth data (no CT scan was available for anyof the patients) of each dataset, we chose to use the2D/3D reconstruction result from manually segmentedcontours as the ‘bronze standard’ (29). The reconstruc-tion result from the automatically segmented contourswas then compared to its associated bronze standard.The open source tool MESH (30) was used to computethe distance between two discrete surface models. Thecolour-coded error distributions of the three datasets areshown in Figure 5 (top, middle and bottom rows, respec-tively). Box-plots of the errors of all three datasets arepresented in Figure 6.

Using the surface models obtained in the secondexperiment, we also measured the morphological dif-ferences between the surface models reconstructed fromthe automatically segmented contours and those from themanually segmented contours. Again, the results com-puted from the surface models reconstructed from themanually segmented contours were taken as the bronzestandard. Three morphological parameters, i.e. the centreof the femoral head, the radius of the femoral head, andthe orientation of the neck axis, were used in this study.The results of this study are shown in Table 2.

In the last experiment, we designed and conducted twostudies on five cadaver datasets to evaluate the accuracy ofreconstructing a surface model from contours extracted bythe present algorithm. The ground truth surface model ofeach cadaver femur was regarded as the ‘gold standard’for accuracy evaluation. In the first study, we ran thepresent contour extraction algorithm on the five-cadaverdataset with the same set of parameters as we used for theclinical datasets to extract the contours of the proximal

Copyright 2009 John Wiley & Sons, Ltd. Int J Med Robotics Comput Assist Surg 2009; 5: 213–222.DOI: 10.1002/rcs

Page 6: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

218 X. Dong and G. Zheng

Figure 4. Results of automatic proximal femur bone contour extraction on clinical datasets. Top row, results on dataset 1. Middlerow, results on dataset 2. Bottom row, results on dataset 3

Table 2. Morphological differences between the surface modelsreconstructed from the automatically segmented contours andthose from the manually segmented contours

Parameters Dataset 1 Dataset 2 Dataset 3 Average

Head centre difference (mm) 2.6 3.5 2.8 3.0Head radius difference (mm) 0.7 1.0 0.5 0.7Neck axis difference (◦) 1.4 6.4 4.8 4.2

femurs from the associated X-ray radiographs. For eachdataset, our automated contour extraction algorithm wasinitialized with manually defined landmarks. We thenperformed 2D/3D reconstructions using the extractedimage contours to obtain patient-specific surface models.The same 2D/3D reconstruction algorithm (10) as weused for the clinical datasets was also used here. Thereconstructed patient-specific surface models were thencompared to their associated gold standard. The opensource tool MESH (30) was used again to computethe error distribution around the reconstructed surface

models. In the second study, we performed similar stepsas we did in the first study, except that we used the methodthat we proposed in (27) to interactively extract contoursof the proximal femurs. Box-plots of the reconstructionerrors of all five cadaver datasets in both studies arepresented in Figure 7, where the top row shows thereconstruction errors when the automated method wasused to extract the image contours and the bottom rowshows the reconstruction errors when the interactivemethod was used. An example of the automated contourextraction and model reconstruction is shown in Figure 8.

Discussion and Conclusions

The results of the first experiment qualitatively demon-strate the efficacy of the proposed approach of automaticcontour extraction, although we did observe that in cer-tain local regions our algorithm failed to find the correctcontours. For this reason, we performed the second and

Copyright 2009 John Wiley & Sons, Ltd. Int J Med Robotics Comput Assist Surg 2009; 5: 213–222.DOI: 10.1002/rcs

Page 7: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

Automatic extraction of proximal femur contours from calibrated X-ray images 219

Figure 5. Colour-coded error distributions when the surface model reconstructed from the automatically segmented contours werecompared to the associated bronze standard. Top row, colour-coded error distribution for dataset 1. Middle row, colour-codederror distribution for dataset 2. Bottom row, colour-coded error distribution for dataset 3

the third experiments to investigate to what degree theerrors in the contour extraction affect the accuracy of thepatient-specific surface model reconstruction.

In the second experiment, when the surface modelsreconstructed from the automatically segmented contourswere compared to their associated bronze standards,which were the surface models reconstructed frommanually segmented contours, local deviations around thehead regions were observed (see Figure 5, Table 2). Thismay be explained by the occlusion caused by the overlapof the femoral head and the pelvis bone, so that the localfeatures of bone contours could not be reliably identified.A possible solution would be to fine-tune the presentapproach for each case, which was not done in the presentstudy. All three cases used the same parameters, eventhough the overall shape of the proximal femur in eachcase was still preserved. The average difference betweenthe surface models reconstructed from the automaticallysegmented contours and those from the manuallysegmented contours models was found to be 1.1 mm.

A similar effect was also observed in the third experi-ment, in which calibrated X-ray radiographs were used.Compared to the gold standard, a mean reconstruction

error of 1.6 mm was found when the automatic methodwas used to extract the image contours. This error wasreduced to 1.0 mm when the interactive method was used.

A further observation in the second and the thirdexperiments was that a traditional Iterative ClosestPoint (ICP)/ASM-like method was used in our 2D/3Dregistration to fit the 3D model with the extractedcontours. Due to the fact that a proper initial alignmentof the 3D model was available, this worked well inour approach. It could be expected that introducing thegraphical model-based Bayesian inference in the processto find the 2D/3D correspondence and the 3D modelparameters further improved its robustness and accuracy.

Based on these experimental results, we suggest usingthe proposed method mainly for intra-operative applica-tions. Although the interactive contour extraction leads tomore accurate results than the automatic one, the latterhas the advantage of eliminating user interactions, whichis important for an intra-operative application, largelydue to the strict sterilization requirement in an operat-ing room. However, when calibrated X-ray radiographsare used, more focus should be placed on reconstruc-tion accuracy, rather than on the elimination of the user

Copyright 2009 John Wiley & Sons, Ltd. Int J Med Robotics Comput Assist Surg 2009; 5: 213–222.DOI: 10.1002/rcs

Page 8: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

220 X. Dong and G. Zheng

Figure 6. Distances between the surface model reconstructed from the automatically segmented contours and those from themanually segmented contours

Figure 7. Errors of reconstructing surface models of all five cadaver femurs in the third experiment when different methods wereused to extract the image contours. Top row, errors of reconstructing all five cadaver femurs when the automated method was usedto extract the image contours. Bottom row, errors of reconstructing all five cadaver femurs when the interactive method was usedto extract the image contours

intervention, due to the pre-operative characteristic ofthe X-ray radiographs. In such a situation, we recom-mend using an interactive or a semi-automatic contourextraction method.

In this paper, we introduced a 3D statistical model-based, fully automatic bone contour extraction frameworkfrom calibrated X-ray images. The automatic initializa-tion is achieved by fitting a simplified multiple com-ponent geometrical 3D model to the observed X-rayimages. The 3D model based initialization algorithm doesnot ask for strict view direction assumption comparedwith 2D model based initialization. The 3D statisti-cal model based bone contour extraction is solved asa simultaneous 2D/3D registration and segmentation.The model-based segmentation is accomplished by a

Bayesian inference procedure, which in principle canoutperform ASM/AAM-based algorithms by simultane-ously optimizing both the global shape constraints andthe local image feature information. Experiments onclinical datasets as well as on cadaver datasets ver-ified the validity and the performance of the presentapproach.

Acknowledgements

The authors would like to thank Mr Schumann for his help inproviding the cadaver datasets and BrainLAB AG for providingthe clinical datasets. The work was partially funded by the SwissNational Science Foundation through the project Co-Me.

Copyright 2009 John Wiley & Sons, Ltd. Int J Med Robotics Comput Assist Surg 2009; 5: 213–222.DOI: 10.1002/rcs

Page 9: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

Automatic extraction of proximal femur contours from calibrated X-ray images 221

Figure 8. An example of automated contour extraction and model reconstruction. Top row, reconstructed surface modelsuperimposed on the input images. Bottom row, apparent contours of the reconstructed model (yellow dots) and the extractedimage contours (white lines)

References

1. Chen Y, Ee X, Leow WK, et al. Automatic extraction of femurcontours from hip X-ray images. In proc. of the firstinternational workshop on Computer Vision for BiomedicalImage Applications (CVBIA 2005), Lecture Notes in ComputerScience, 2005; 3765: 200–209.

2. de Luis-Garcia R, Martin-Fernandez M, Arribas JI, et al. A fullyautomatic algorithm for contour detection of bones in handradiographs using active contours. In proc. of the 2003International Conference on Image Processing, 2003; part III:421–424.

3. Roberts MG, Cootes TF, Adams JE. Automatic segmentation oflumbar vertebrae on digitized radiographs using linked activeappearance models, Medical Image Understanding and Analysis2006; part I: 120–124.

4. Gregory JS, Waarsing JH, Day J, et al. Early identification ofradiographic osteoarthritis of the hip using an active shapemodel to quantify changes in bone morphometric features:can hip shape tell us anything about the progression ofosteoarthritis? Arthrit Rheumat 2007; 56(11): 3634–3643.

5. Gregory JS, Stewart A, Undrill PE, et al. Bone shape, structure,and density as determinants of osteoporotic hip fracture: a pilotstudy investigating the combination of risk factors. Invest Radiol2005; 40(9): 591–597.

6. Gottschling H, Roth M, Schweikard A, et al. Intraoperative,Fluoroscopy-based planning for complex osteotomies of theproximal femur. Int J Med Robotics Comput Assist Surg 2005;1(3): 67–73.

7. Bartolini F, Carfagni M, Governi L. Model-based extraction offemoral medulla ducts from radiographic images. Image VisionComput 2004; 22: 173–182.

8. Tian TP, Chen Y, Leow WK, et al. Computing neck–shaft angleof femur for X-ray fracture detection. The 10th internationalconference on Computer Analysis of Images and Patterns (CAIP2003), Lecture Notes in Computer Science 2003; 2756: 82–89.

9. Benameur S, Mignotte M, Parent S, et al. 3D/2D Registrationand segmentation of scoliotic vertebrae using statistical models.Comput Med Imag Graphics 2003; 27(5): 321–337.

10. Zheng G, Gonzalez Ballester MA, Styner M, et al. Reconstructionof patient-specific 3D bone surface from 2D calibratedfluoroscopic images and point distribution model. The 9thinternational conference on Medical Image Computing and

Computer-Assisted Intervention (MICCAI 2006), Part I, LectureNotes in Computer Sciences 2006; 4190: 25–32.

11. Lamecker H, Wenckebach TH, Hege H-C. Atlas-based 3D-shapereconstruction from X-ray images. The 18th InternationalConference on Pattern Recognition (ICPR 2006); part I:371–374.

12. Tang TS, Ellis RE. 2D/3D deformable registration using a hybridatlas. The 8th international conference on Medical ImageComputing and Computer-Assisted Intervention (MICCAI 2005),Lecture Notes in Computer Science 2005; 3750: 223–230.

13. Behiels G, Vandermeulen D, Maes F, et al. Active shape model-based segmentation of digital X-ray images. The 2ndinternatioanl conference on Medical Image Computing andComputer-Assisted Intervention (MICCAI 1999), Lecture Notesin Computer Science 1999; 1679: 128–137.

14. Howe B, Gururajan A, Sari-Sarraf H, et al. Hierarchicalsegmentation of cervical and lumbar vertebrae using acustomized generalized Hough transform and extensions toactive appearance models. Proceedings of the IEEE 6thSouthwest Symposium on Image Analysis and Interpretation,March 2004; 182–186.

15. Langs G, Peloschek P, Bischof H. Determining position and fineshape detail in radiological anatomy; pattern recognition.Proceedings of the 25th Annual Symposium of the GermanAssociation for Pattern Recognition. Lecture Notes in ComputerScience 2003; 2781: 532–539.

16. Seise M, McKenna SJ, Ricketts IW, et al. Probabilisticsegmentation of the knee joint from X-ray images. MedicalImage Understanding and Analysis 2006; 110–114.

17. de Bruijne M, Nielsen M. Image segmentation by shapeparticle filtering. The 17th International Conference on PatternRecognition (ICPR 2004); part III.

18. Zheng G, Rajamani KT, Nolte L-P. Use of a dense surfacepoint distribution model in a three-stage anatomical shapereconstruction from sparse information for computer assistedorthopaedic surgery: a preliminary study. The 7th AsianConference on Computer Vision (ACCV 2006). Lecture Notesin Computer Science 2006; 3852: 52–60.

19. Ma B, Ellis RE. Surface-based registration with a particle filter.The 7th international conference on Medical Image Computingand Computer-Assisted Intervention (MICCAI 2004), LectureNotes in Computer Science 2004; 3216: 566–573.

20. Lee MW, Cohen I. Human upper body pose sstimation in staticimages. The 8th European Conference on Computer Vision

Copyright 2009 John Wiley & Sons, Ltd. Int J Med Robotics Comput Assist Surg 2009; 5: 213–222.DOI: 10.1002/rcs

Page 10: Automatic extraction of proximal femur contours from calibrated X-ray images using 3D statistical models: an in vitro study

222 X. Dong and G. Zheng

(ECCV 2004). Lecture Notes in Computer Science 2004; 3022:126–138.

21. Sigal L, Bhatia S, Roth S, et al. Tracking loose-limbed people.The 2004 IEEE Computer Society Conference on ComputerVision and Pattern Recognition (CVPR 2004) part 1: 421–428.

22. Wu Y, Hua G, Yu T. Tracking articulated body by dynamicMarkov network, The 9th IEEE International Conference onComputer Vision (ICCV 2003) 1: 1094–1101.

23. Coughlan J, Ferreira S. Finding deformable shapes using loopybelief propagation. The 7th European Conference on ComputerVision (ECCV 2002). Lecture Notes in Computer Science 2002;2352: part III: 453–468.

24. Rangarajan A, Coughlan J, Yuille AL. A Bayesian networkframework for relational shape matching. The 9th IEEEInternational Conference on Computer Vision (ICCV 2003)671–678.

25. Barrett AR, Davies BL, Gomes MP, et al. Preoperative planningand intraoperative guidance for accurate computer-assistedminimally invasive hip resurfacing surgery. Proc Inst Mech EngH J Eng Med 2006; 220(7): 759–773.

26. Schumann S, Zheng G, Nolte L-P. Calibration X-ray radiographsand its feasible application for 2D/3D reconstruction of the

proximal femur. In proc. of the 30th Annual InternationalConference of the IEEE Engineering in Medicine and BiologySociety (IEEE EMBC’08), 2008; 470–473.

27. Zheng G, Schumann S. 3D reconstruction of a surface modelof the proximal femur from digital biplanar radiographs. Inproc. of the 30th Annual International Conference of the IEEEEngineering in Medicine and Biology Society (IEEE EMBC’08),2008; 66–69.

28. Dong X, Zheng G. Fully automatic determination ofmorphological parameters of proximal femur from calibratedfluoroscopic images through particle filtering. The 3rdInternational Conference on Image Analysis and Recognition(ICIAR 2006), Lecture Notes in Computer Science 2006; 4142:535–546.

29. Jannin P, Fitzpatrick JM, Hawkes DJ, et al. Validation of medicalimage processing in image-guided therapy. IEEE Trans Med Imag2002; 21(12): 1445–1449.

30. Aspert N, Santa-Cruz D, Ebrahimi T. MESH: measuring errorsbetween surfaces using the Hausdorff distance. InternationalConference on Multimedia and Expo (ICME) 2002; 1: 705–708.

Copyright 2009 John Wiley & Sons, Ltd. Int J Med Robotics Comput Assist Surg 2009; 5: 213–222.DOI: 10.1002/rcs