RESEARCH ARTICLE OPEN ACCESS Assessment of …Index Terms—Bone Age assessment, Statistical Appearance Models, Groupwise Registration, Parts + Geometry Models (P+G). F 1 INTRODUCTION

INTERNATIONAL JOURNAL OF COMPUTER TECHNIQUES, VOL. 4, NO. 6, DECEMBER 2017 47

RESEARCH ARTICLE OPEN ACCESS

Assessment of Efficiency of Part & GeometryModels for Groupwise Registration

Steve A. Adeshina,1∗ and Timothy F. Cootes 2∗

1Department of Electrical & Electronics Engineering,Nile University of Nigeria, Abuja,E-mail: [email protected]

2Center for Imaging Sciences, The University of Manchester, U.K

Abstract—We evaluate the performance of a system which addresses the problem of building detailed models of shapeand appearance of complex structures, given only a training set of representative images and some minimal manualintervention. We focus on objects with repeating structures (such as bones in the hands), which can cause normaldeformable registration techniques to fall into local minima and fail. Using a sparse annotation of a single image we canconstruct a parts+geometry (P+G) model capable of locating a small set of features on every training image. Iterativerefinement leads to a model which can locate structures accurately and reliably. The resulting sparse annotations aresufficient to initialise a dense groupwise registration algorithm, which gives a detailed correspondence between allimages in the set. We demonstrate the method on a much larger set of radiographs of the hand while comparing resultswith that of the earlier work, we achieved a sub-millimeter accuracy in a prominent group.

Index Terms—Bone Age assessment, Statistical Appearance Models, Groupwise Registration, Parts + GeometryModels (P+G).

F

1 INTRODUCTION

Many forms of model can be constructed if wehave accurate correspondences defined across aset of training images. However, obtaining suchcorrespondences can be difficult and time con-suming. In most early work on statistical shapemodels, for instance [4], the correspondenceswere created manually. More recently therehas been considerable research into automatedmethods of achieving correspondence, such asfrom boundaries in 2D or surfaces in 3D (eg[6]), or more generally by directly registeringimages using non-rigid registration methods or‘groupwise’ techniques [3].

In our earlier paper we tackled the prob-lem of registering images of objects with con-siderable shape variation and multiple similarsub-parts. The key problem with such data is

Manuscript received November 24, 2017; revised Decemebr 1, 2017.

one of initialisation. A common approach togroupwise registration is to first find an affinetransformation which gives an approximate so-lution, then perform non-rigid registration toan evolving mean to obtain more exact results[3]. Unfortunately, with the degree of variabil-ity exhibited in the hands, the affine stage isinsufficient.

We use a parts+geometry model [9]. Thelocal geometry can be used to efficiently selectbetween multiple candidates for the parts. Don-ner et al. demonstrated how a sophisticatedparts + geometry model can accurately locatepoints in such images and how such a modelcan be constructed automatically from a set ofimages in which only one is manually anno-tated [8]. However, the method was only eval-uated on a small set of 12 hand radiographs.

In this paper we show how a simple parts +geometry model can be learned from a large set

International Journal of Computer Techniques, Vol. 4, No. 6, December 2017

of images using only one manually annotatedimage and how this can be used to initialisea groupwise registration algorithm, leading todense correspondences [1]. We extend our ear-lier work to deal with 536 images (as opposedto 94). The key problem is the huge variationthat exist in registering radiographs of childrenand young adults for automatic determinationof skeletal maturity. This makes the originalmethod perform less effectively.

In the following we describe the techniquein tackling the inherent variation, demonstrateits use and evaluate it by comparing the resultswith the initial work [1].

2 RELATED WORK

Model constructions could be easy if we can es-tablish accurate correspondences across a set oftraining images. Unfortunately, obtaining suchcorrespondences can be difficult and time con-suming. In most early works on statistical shapemodels, for instance [4], the correspondenceswere created manually. More recently therehas been considerable research into automatedmethods of achieving correspondence, such asfrom boundaries (eg [5]) in 2D or surfacesin 3D (eg [6]), or more generally by directlyregistering images using non-rigid registrationmethods [11] or ‘groupwise’ techniques [2], [3],[14], [15].

The natural approach is thus to use aParts+Geometry model [13], [9], [10]. The lo-cal geometry can be used to efficiently selectbetween multiple candidates for the parts. Forinstance, Donner et al. [7] demonstrated howa sophisticated parts + geometry model can ac-curately locate points in such images. In furtherwork [8], they showed that such a model can beconstructed automatically from a set of imagesin which only one is manually annotated. How-ever, the method was only evaluated on a smallset of 12 hand radiographs. In related work,Langs et al. [12] describe a method of con-structing sparse shape models from unlabeledimages, by finding multiple interest points andusing an MDL approach to determine optimalcorrespondences.

We show how a simple parts + geometrymodel can be learned from a large set of imagesusing only one manually annotated image. Weconstruct an initial model from the single imageand use it to search the rest of the set. The bestresults are then used to update the model in aniterative scheme. The result is a trained model,together with a sparse set of correspondencesacross the set. These correspondences can thenbe used to initialise a group based registrationalgorithm, leading to dense correspondences.Figure 1 shows the process diagram for thesteps described above.

Fig. 1. P+G models and groupwise registration process diagram

3 METHODS

3.1 Data Set

We have access to a database of radiographs ofthe non-dominant hand of normally developingchildren. The children were enrolled on a boneageing study at the University of Manchester.Their ages ranged between 5 years and 20years. In the following work we used a subsetof 170 (87 male and 83 female) digitized radio-graphs of normal children.

3.2 Multi-Resolution Patch Models

Given one or more training images in which aparticular region has been annotated, we canconstruct a statistical model of the region. Weassume that the region is of fixed shape, butmay vary in size and orientation. In the sim-plest case the region is an oriented rectangle orellipse, centred on a point, p with scale s andorientation θ.

If g(t) are the intensities sampled fromn pixels in the region with pose parameterst = {p, s, θ}, normalised to have a mean of zero

ISSN :2394-2231 http://www.ijctjournal.org page 48


and unit variance, then the quality of fit to amodel is evaluated as

fi(g(t)) =n∑

j=1

|gj − gij|/σij (1)

where gi is the vector of mean intensities for theregion and σij is an estimate of the mean abso-lute difference from the mean across a trainingset.1

We can then search new images with sucha model, by performing an exhaustive searchat a range of positions, orientations and scalesto locate local minima of fi(g(t)). This result inmultiple responses for each patch [1].

3.3 Geometric Relationships

To disambiguate the multiple responses of asingle patch model, we create a model con-taining a set of N patch models, together witha model of the pairwise relationships betweenthem. This is a widely used and effective tech-nique [9].

Given multiple possible candidates for eachpart position (from the patch detectors), weused a graph algorithms to locate the opti-mal solutions. We used a variant of dynamicprogramming in which a network is createdwhere each node can be thought of as havingat most two parents. Details of this method arediscussed in [1].

Each candidate response for part i has apose with parameters ti = {pi, si, θi}. The re-lationship between part i and part j can berepresented in the cost function, fij(ti, tj). Thiscan be derived from the joint PDF of the param-eters.

In the following we take advantage of thefact that the orientation and scale of the objectsare approximately equivalent in each image,and simply use a cost function based on therelative position of the points:

fij(ti, tj) = ((pj −pi)−dij)TS−1ij ((pj −pi)−dij)

(2)

1. We find this form (which assumes the data has an expo-nential distribution) gives more robust results than normalisedcorrelation, which is essentially a sum of squares measure.

where dij is the mean separation of the twopoints, and Sij is an estimate of the covariancematrix.

The matching algorithm thus seeks to findthe candidates which minimise the followingfunction

F =N∑i=1

fi(gi) + α∑

(i,j)∈Arcs

fij(pi,pj) (3)

The value of α affects the relative impor-tance of patch and geometry matches. In thefollowing we use α = 0.1, chosen by prelimi-nary experiments on a small subset of the data.Ways of automatically choosing a good value ofα are the focus of current research.

3.4 Building the ModelWe initialise a model using a set of parts de-fined by boxes placed on a single image bythe user (for instance, the rectangles shown inFigure 2a). This takes about one minute to do,and allows the algorithm to take advantage ofuser supplied knowledge. We then automati-cally define a set of connecting arcs based onthe distances between the centres of the boxes.We use a variant of Prim’s algorithm for theminimum spanning tree, where each node hastwo parent nodes, rather than one [1].

We then refine the model by applying it tothe whole dataset, ranking the results by finalfit value (per image), and building statisticalmodels of intensity and pairwise relationshipfrom the best 50% of the matches.

3.5 Dense CorrespondenceAt convergence we obtain a model of parts andgeometry, together with a sparse annotation ofevery image in the training set. The centresof each part region define correspondences.Weuse these to initialise a groupwise registration.We place a dense mesh of control points onthe first image, use a thin-plate spline basedon the sparse annotation to propagate thesepoints to all other images. We then computethe mean shape and warp each example intothe mean. Furthermore we perform non-rigidregistration [3] to modify the control points oneach image to best match to the mean. Finallywe re-compute the mean and iterate.



4 EXPERIMENTS

We applied the technique described above to aset of 536 radiographs of the hands of children,taken as part of another study2. We dividedthe dataset into three age-groups. AgeGroup1-63 images (5 - 7 yrs), AgeGroup2 -284 images(8-13 yrs) and AgeGroup3 - 189 images (14 -19 years) In our earlier work [1] we foundthe optimal number of boxes to be 19 boxes.These 19 boxes were annotated on one image(see Figures 2a). For each choice of boxes on asingle image, a model of parts and geometrywas constructed and used to locate equivalentpoints on other images. The models were thenrebuilt and refined as described above. Figure2a shows the initial 19 boxes on one of theimages, together with the automatically cho-sen connectivity. Matches with the final modelare shown in Figure 2b,c,d,e for the variousgroups and an example of failure in 2f. Thefound points in each of the groups were usedto initialise a groupwise algorithm as describedabove. Qualitative results of the registration isshown in Figures 3. The crispness of the imagesindicate a good alignment.

a) b) c)

d) e) f)

Fig. 2. Example of model(a), search results with 19 parts forset94(b) [1], AgeGroup1(c), AgeGroup2 (d), AgeGroup3 (e) andan example of a failure (f) respectively (see the tip of the fifthfinger near the label).

We evaluated the accuracy of the points lo-cation by comparing with manual annotations

2. The authors would like to thank K.Ward, R.Ashby, Z.Mughal and Prof.J.Adams for providing the images.

a) set94 b) 5-7yr

c) 8-13yr d) 14-19yr

Fig. 3. Final mean images after groupwise registration. a)set94 [1], b) AgeGroup1, c)AgeGroup2 and c) AgeGroup3.

based on an evaluation framework formulatedin [1]. The mean distance errors for sparsepoint errors was found to be 0.70 ± 0.08mm,1.08 ± 0.18mm, 0.91 ± 0.15mm, 0.75 ± 0.09mmfor the set94 (images used in [1] ), AgeGroup1,AgeGroup2, AgeGroup3 respectively. The re-sult of AgeGroup3 14 -19, a very difficult group,is comparable to the original result obtained in[1]. Figure 4a presents the distribution of theerrors and compare the various groups. For thedense correspondence accuracy, a median errorof 0.94mm, 1.38mm, 1.1mm and 1.01mm forthe set94, AgeGroup1, AgeGroup2, AgeGroup3respectively. These errors are higher than insparse point placement because the evaluationis based on the entire image region [1]. Figure4b presents the distribution of the errors andcompare the various groups. Note that in bothcases errors are highest for AgeGroup1. Thefew number of images and very large variationmay be responsible. Sometimes there is no cor-respondence amongst the bones.

5 DISCUSSION AND CONCLUSIONS

We have evaluated an approach for automat-ically locating sparse correspondences acrossa set of images, by constructing a parts and



a)

b)

Fig. 4. Comparison of statistics of points errors for variousgroups. a) Accuracy of sparse point placement and b) Errorsafter groupwise registration (mm).

geometry model with an extended dataset. Weachieve an accuracy of 0.75mm on the posi-tioning of the chosen parts. This is significantlybetter than results quoted by Donner et al.[8](approx. 1.5mm, though on a different, smallerdataset). The found points are sufficient to ini-tialise a more detailed group-wise registrationwhich can give dense point correspondenceswith approximately 1mm accuracy over thewhole hand. We can conclude that these re-sults are comparable with our earlier work [1].We have commenced more work on the Age-Group1 to achieve higher accuracy.

ACKNOWLEDGMENTS

The authors would like to thank Prof JudithAdams for providing the Radiograph images..

Steve A. Adeshina Dr Steve A Adeshina is with the Nile Uni-versity of Nigeria, Abuja Nigera.

Timothy F. Cootes Prof Tim Cootes is with Centre for ImagingSciences, The University of Manchester, Manchester, UK.

REFERENCES

[1] S. A. Adeshina and T. F. Cootes. Constructing part-based models for groupwise registration. In Proc. IEEEInternational Syposium on Biomedical Imaging, 2010.

[2] S. Baker, I. Matthews, and J.Schneider. Automatic con-struction of active appearance models as an image codingproblem. IEEE Trans. on Pattern Analysis and MachineIntelligence, 26(10):1380–84, 2004.

[3] T. Cootes, C. Twining, V.Petrovic, R.Schestowitz, andC. Taylor. Groupwise construction of appearance mod-els using piece-wise affine deformations. In 16th BritishMachine Vision Conference, volume 2, pages 879–888, 2005.

[4] T. F. Cootes, C. J. Taylor, D. Cooper, and J. Graham. Activeshape models - their training and application. ComputerVision and Image Understanding, 61(1):38–59, Jan. 1995.

[5] R. Davies, C.Twining, T. Cootes, and C. Taylor. A min-imum description length approach to statistical shapemodelling. IEEE Trans. on Medical Imaging, 21:525–537,2002.

[6] R. Davies, C.Twining, T. Cootes, J. Waterton, and C. Taylor.3D statistical shape models using direct optimisation ofdescription length. In European Conference on ComputerVision, volume 3, pages 3–20. Springer, 2002.

[7] R. Donner, B. Micusik, G. Langs, and H. Bischof. SparseMRF appearance models for fast anatomical structurelocalisation. In Proc. British Machine Vision Conference,volume 2, pages 1080–89, 2007.

[8] R. Donner, H. Wildenauer, H. Bischof, and G. Langs.Weakly supervised group-wise model learning based ondiscrete optimization. In Proc. MICCAI, volume 2, pages860–868, 2009.

[9] P. Felzenszwalb and D.P.Huttenlocher. Pictorial structuresfor object recognition. International Journal of ComputerVision, 61(1):55–79, 2005.

[10] R. Fergus, P. Perona, and A. Zisserman. A visual categoryfilter for google images. In 8th European Conference onComputer Vision, volume 1, pages 242–256. Springer, 2004.

[11] A. Frangi, D. Rueckert, J. Schnabel, and W. Niessen. Au-tomatic construction of multiple-object three-dimensionalstatistical shape models: Application to cardiac modeling.IEEE Trans. Medical Imaging, 21:1151–66, 2002.

[12] G.Langs, R.Donner, Peloschek, and H.Bischof. Robustautonomous model learning from 2D and 3D data sets.In Proc. MICCAI, volume 1, pages 968–976, 2007.

[13] M.A.Fischler and R.A.Elschlager. The representation andmatching of pictorial structures. IEEE Trans. Computer,22(1):67–92, 1973.

[14] M. Miller. Computational anatomy: shape, growth, andatrophy comparison via diffeomorphisms. NeuroImage,23:S19–S33, 2004.

[15] S.Joshi, B.Davis, M.Jomier, and G.Gerig. Unbiased dif-feomorphic atlas construction for computational anatomy.NeuroImage, 23:S151–S160, 2004.


Documents

RESEARCH ARTICLE OPEN ACCESS Assessment of …Index Terms—Bone Age assessment, Statistical Appearance Models, Groupwise Registration, Parts + Geometry Models (P+G). F 1 INTRODUCTION