Transcript
Page 1: Validation of automatic bone age determination in children with congenital adrenal hyperplasia

ORIGINAL ARTICLE

Validation of automatic bone age determinationin children with congenital adrenal hyperplasia

David D. Martin & Katharina Heil & Conrad Heckmann &

Angelika Zierl & Jürgen Schaefer & Michael B. Ranke &

Gerhard Binder

Received: 7 September 2011 /Revised: 6 April 2013 /Accepted: 22 April 2013 /Published online: 5 October 2013# Springer-Verlag Berlin Heidelberg 2013

AbstractBackground Determination of bone age is routinely used forfollowing up substitution therapy in congenital adrenalhyperplasia (CAH) but today is a procedure with significantsubjectivity.Objective The aim was to test the performance of automaticbone age rating by the BoneXpert software package in allradiographs of children with CAH seen at our clinic from1975 to 2006.Materials and methods Eight hundred and ninety-two left-hand radiographs from 100 children aged 0 to 17 years werepresented to a human rater and BoneXpert for bone agerating. Images where ratings differed by more than 1.5 yearswere each rerated by four human raters.Results Rerating was necessary in 20 images and the reratingresult was closer to the BoneXpert result than to the originalmanual rating in 18/20 (90 %). Bone age rating precisionbased on the smoothness of longitudinal curves comprising atotal of 327 data triplets spanning less than 1.7 years showedBoneXpert to be more precise (P<0.001).Conclusion BoneXpert performs reliable bone age ratings inchildren with CAH.

Keywords Congenital adrenal hyperplasia . Skeletalmaturation . Bone age rating . Glucocorticoid substitution

Introduction

Congenital adrenal hyperplasia (CAH) is the generic term fora group of autosomal recessive metabolic diseases involvinga deficiency of any of five enzymes responsible for thesynthesis of cortisol in the adrenal gland. The most frequentform of CAH, accounting for more than 90 % of all cases, isattributable to a deficiency of the 21-hydrolase enzyme [1].Life-endangering “salt-wasting syndrome” can ensue if bothcortisol and aldosterone synthesis are affected. The variant,which only affects cortisol synthesis, is referred to as “simplevirilizing CAH.” It manifests as virilization of variabledegree in girls and precocious puberty in boys. Milder, non-classic forms of CAH often remain undetected in males andlead to hirsuitism and impaired fertility in females [2]. Theworldwide incidence of CAH has been estimated at 1 in16,000 births [3].

If left untreated, the overproduction of androgens in CAHalso leads to accelerated gain in height. However, this effectis overlaid with accelerated maturation of the skeleton,resulting in short adult stature in most children [4–8].Therefore, besides avoiding an Addisonian crisis, attainmentof normal adult stature is one of the primary goals in thetreatment of CAH. This primarily consists of providing thepatient with enough glucocorticoids and mineralocorticoidsto suppress ACTH-mediated excess androgen production.Finding the correct dosage poses a challenge because evenslight overdoses of glucocorticoids will curb growth overtime [6, 8–11]. Ameta-analysis of 18 studies carried out from1977 to 1997 found a mean adult height followingglucocorticoid treatment of −1.37 SD below the respectivepopulation mean [10]. Experimental therapeutic approachesinclude addition of growth hormone [12] or peripheralinhibition of androgen activity and estrogen production.

D. D. Martin (*) :K. Heil :C. Heckmann :M. B. Ranke :G. BinderPediatric Endocrinology and Diabetology, University Children’sHospital, Hoppe-Seyler Strasse 1, 72076 Tuebingen, Germanye-mail: [email protected]

A. Zierl : J. SchaeferDepartment of Diagnostic Radiology, University of Tuebingen,Hoppe- Seyler-Str. 3, 72076 Tuebingen, Germany

Pediatr Radiol (2013) 43:1615–1621DOI 10.1007/s00247-013-2744-8

Page 2: Validation of automatic bone age determination in children with congenital adrenal hyperplasia

Treatment of children with CAH involves regular mea-surement of 17-hydroxyprogesterone as well as bone age andheight determination [1]. Some authors consider bone age tobe the most useful follow-up parameter after body height,partly due to the fact that 17-hydroxyprogesterone respondsquickly to medication, thus giving no indication of a patient’slong-term compliance.

Rating bone age in CAH can require greater thanaverage expertise, especially when bone age is extremelyadvanced. In times of demands for higher productivity inradiology, radiologic expertise is becoming an increasinglyprecious resource and the advance of computerized bone agerating methods is therefore welcomed by many as a potentialmeans of relieving radiologists from analyzing large numbersof unremarkable radiographs. The CE (European Conformi-ty)-marked medical device used for this purpose is BoneXpert(Visiana, Holte, Denmark), a software package that can beused on one image at a time in daily routine or, alternatively, toanalyze large quantities of hand radiographs in an unattendedbatch job and whose rating reliability in healthy populationshas been demonstrated [13–16]. Furthermore, the automatedmethod has been successfully used for children with shortstature of various diagnoses [17] or with central precociouspuberty [18]. However, it has not to our knowledge beenvalidated in children exposed to androgens since earlygestation, such as in the case of children with CAH. Further,our cohort of children with CAH includes some children withextremely advanced bone age due to the fact that they hademigrated from countries where screening for CAH was notinstitutionalized at the time of their birth. These children areparticularly challenging in terms of management, starting withtheir bone age reading. We were interested to test whether theautomated method could cope with radiographs from suchchildren.

The purpose of this study was to determine howautomated bone age performs on radiographs of childrenwith CAH compared with human bone age raters.

Materials and methods

Eight hundred and ninety-two left-hand radiographs from100 children and adolescents aged 0 to 17 years with adiagnosis of CAH who had been treated at our clinic duringthe period from January 1, 1975, to December 31, 2006, wereincluded in the study. If the films were not already availablein digital form as DICOM images, they were scanned with aVidar Diagnostic Pro Advantage scanner (Vidar, Hemdon,VA). The study was approved by the Tübingen UniversityHospital Ethics Committee.

Automatic bone age ratingwas performed usingBoneXpertversion 1.0 (Visiana, Holte, Denmark, www.BoneXpert.com).BoneXpert calculates bone age based on the shape and

appearance of 13 bones of the hand: the phalanges andmetacarpals of the first, third and fifth ray and the radius andulna. The program rejects a bone if its shape is abnormal or ifits bone age deviates by more than 2.4 years from the mean ofall bones. Furthermore, if fewer than 8 bones are accepted, theentire radiograph is rejected for bone age analysis. TheintendedGreulich-Pyle bone age range for the automatic ratingin its current version is 2 to 15 years for girls, and 2.5 to17 years for boys. A more detailed description of thecalculation methods employed by BoneXpert has beenpublished elsewhere [17, 19].

In 35 examinations, the image was available both as aDICOM file and as a printout on film of the DICOM image. Acomparison of the ratings performed by automated bone ageon the DICOM files and on the scanned film-printoutsyielded differences no greater than 0.5 years, of which 18were greater than 0.2 years. The mean difference was0.0 years, i.e. there was no trend for the scanned films to yieldolder or younger bone age values than the DICOM images.The DICOM file was used whenever one was available.

Validity

Since an objective measure of bone age does not exist,we must resort to various indirect ways of assessing thevalidity (accuracy) of a new rating method. Thodberget al. [15] circumvented the problem of the lack of anobjective bone age by judging the accuracy of bone agerating on the basis of its ability to predict an objectiveparameter to which it is related, namely final height.However, this approach is not recommendable in childrenwhose final height may be affected by subsequent treatment.In this paper, we look for deviations from the resultsobtained with the manual method in terms of bias andstandard deviation. This is an example of analysis ofagreement between two measurements, so it was naturalto use Bland-Altman plots and – in accordance with thisconcept – to use the standard deviation rather than thecorrelation. It was tested whether the bias is significantlydifferent from zero (a qualitative result), while the standarddeviation was reported as a quantitative endpoint.

Notice that the standard deviation of the differencesbetween the two methods is usually larger than the standarddeviation from the line of fit because it ignores the bone age-related slope between the two methods in a Bland-Altmanplot.

An additional, quantitative test was included as follows:Images for which BoneXpert bone age (BXBA) and manualbone age (ManBA) differed by more than 1.5 years werererated by four experienced raters. Rerating was donewithout knowledge of ManBA, BXBA and chronologicalage. The mean of the four bone age values, referred to as theReferenceBA, was compared anewwith BXBA andManBA.

1616 Pediatr Radiol (2013) 43:1615–1621

Page 3: Validation of automatic bone age determination in children with congenital adrenal hyperplasia

The same approach has been used previsouly [17, 18].ReferenceBA acts as a secondary and very reliable outcome.The performance of BXBA and ManBA in terms of theirdeviation from ReferenceBA was calculated using the 1-sample proportions test with continuity correction, takinginto account that only observations from different subjectswere statistically independent.

Precision

By contrast, there is no need for an objective measure ofbone age to determine the precision of a new bone agerating method, i.e. its ability to generate reproducibleresults. One way to determine this is to study thesmoothness of longitudinal curves obtained. In thepresent study, the precision of automatic and manualrating was assessed in terms of the smoothness oflongitudinal curves using the triplet method [19]. Thisinvolves breaking down individual longitudinal bone ageseries with n bone age measurements into n – 2 tripletsof consecutive bone age measurements and consideringthe residual between the middle bone age and the linearinterpolation between the two measurements on eitherside, for each triplet. The triplet method assumes thatthe three bone age measurements lie on a straight line asa function of age if there is no precision error. Hence,any deviation from the line is interpreted as due toprecision error of the bone age method. Since in realitymany more factors lead to a deviation from a straightline, the triplet method yields an upper limit of the trueprecision. To improve the estimate, we only consideredtriplets spanning less than 1.7 years. This left us with327 triplets out of the original 604 (54 %). The result ofthe precision analysis is both quantitative and qualita-tive. The quantitative result was the estimated value of theprecision of the automated and manual methods given withconfidence intervals. In addition,we compared the precision ofmanual and automated methods and tested whether they weresignificantly different, i.e. a qualitative result.1

To further illustrate the behaviour and precision ofautomated bone age compared with manual bone agerating in children with extremely deviant bone age, weplotted the longitudinal course of three children whoseskeletal maturity was particularly advanced. For this weselected, from the subset of children of whom at least12 images were available, the three with the most

advanced bone age (mean of ManBA and BXBA minusCA) at their first visit.

Statistical calculations were performed using the JMP 9software package (SAS Institute Inc., Cary, NC) and version2.12.1 of the R statistics software (www.r-project.org).

Results

Analysis of rejected images

One hundred sixteen of the 892 images were rejected byBoneXpert. In 111 of these, the rejection was due to bone agebeing below BoneXpert’s specified rating range, i.e. ManBAbelow 2.0 years in girls or below 2.5 years in boys. For theremaining five images, three were due to poor image quality,one to improper scanning, and one remains unexplained andthus represents the inefficiency of the automated method.

Comparison between ManBA and BXBA

For the 776 images (480 from girls, 296 from boys; Table 1)analyzed by automated bone age, the mean difference BXBA–ManBAwas −0.02 years (N.S.), the slope of the line of fit ina Bland-Altman plot was negligible at −0.02 years/year andthe standard deviation (SD) of the signed differences was0.72 years. The mean of the absolute (unsigned) differenceswas 0.54 years (SD 0.40 years). In 20 images, the absolutedifference between BXBA and ManBA was greater than1.5 years (Fig. 1).

These images were submitted for blind rerating by fourraters (two radiologists and two pediatric endocrinologists –all having between 10 and 30 years of practice in bone agerating). None of the rerated images showed an absolutedifference between ReferenceBA and BXBA greater than1.5 years. ReferenceBAwas closer toManBA in 2 images andcloser to BXBA in 18 (Table 2). To estimate the statisticalsignificance of this observed advantage ofBXBA,we note thatthe 20 rerated images are from ten children, so we have tenrather than 20 independent observations, and the two caseswhere ManBA is better than BXBA occur in children withthree visits, so for these two children, ManBAwas better thanBXBA in one-third of the visits. Thus, we have observedBXBA to be better thanManBA in 9.33/10 independent cases

Table 1 Characteristics at the time of the first radiograph (childrenwhose radiographs were accepted by BoneXpert): mean chronologicalage and manual bone age (ManBA) by gender

Gender Number ofchildren

Mean age ± SD(y)

Mean ManBA ± SD(y)

Boys 41 5.0 ± 3.4 7.2 ± 5.3

Girls 52 4.3 ± 3.2 5.0 ± 3.4

Pediatr Radiol (2013) 43:1615–1621 1617

1 The triplets formed from a child are not statistically independentbecause each visit can participate in up to three triplets. The confidenceinterval (CI) of the estimated precision was therefore estimated by aMonte Carlo technique, which showed that the CIs are a factor of 1.3times larger than one would derive by assuming that the triplets wereindependent

Page 4: Validation of automatic bone age determination in children with congenital adrenal hyperplasia

(93 %). We now take as null hypothesis that ManBA and BXare equally close to theReferenceBA and a proportion test thenshows that BXBA is closer to ReferenceBA than is ManBAwithP=0.02. (The test for the proportion 1 in 10 givesP=0.027and for 0 in 10 gives P=0.004, and the quoted p-value is theinterpolation to the proportion 0.67 in 10.) Notice also that in

this computation of statistical significance it is irrelevant howmany images and subjects there were in the total study, prior toselecting the disputed cases.

Our analysis of bone age rating precision based on thesmoothness of longitudinal curves comprised a total of 327data triplets spanning less than 1.7 years for bothManBA and

Table 2 Bone age rating results for the 20 images with a difference between the original manual rating (ManBA) and automatic rating (BXBA)greater than 1.5 years. These were subsequently rerated (ReferenceBA)

ID Image no. Gender Age (y) Bone age ratings (y) Bone age rating differences (y) #

ReferenceBA ManBA BXBA ManBA-ReferenceBA BXBA-ReferenceBA BXBAworse §

3187 7 boy 5.7 10.1 11.5 9.4 1.4 −0.7

3187 9 boy 7.0 11.2 12.5 10.8 1.3 −0.4

3242 7 boy 6.3 10.0 11.5 9.6 1.5 −0.4

3242 9 boy 7.2 9.9 12.8 9.6 2.9 −0.3

3752 6 boy 9.9 14.7 17.0 14.3 2.3 −0.4

4326 16 girl 14.4 13.5 15.0 13.3 1.5 −0.2

5131 11 girl 6.7 6.8 5.0 6.6 −1.8 −0.2

5131 14 girl 8.9 10.0 8.8 10.5 −1.2 0.5

5818 9 girl 5.9 5.4 4.5 6.4 −0.9 1.0 X

5818 10 girl 6.5 6.7 5.0 6.8 −1.7 0.1

5818 20 girl 11.6 10.9 9.8 11.5 −1.2 0.6

5989 7 girl 14.3 16.8 18.0 15.9 1.2 −0.9

7653 1 boy 2.4 5.3 2.8 5.8 −2.6 0.5

7653 3 boy 7.8 6.3 4.5 6.9 −1.8 0.6

7653 4 boy 9.1 7.0 5.8 7.4 −1.3 0.4

7653 5 boy 10.3 7.3 6.3 7.9 −1.1 0.6

8253 3 boy 5.6 12.9 14.0 12.4 1.1 −0.5

9516 2 girl 2.6 5.4 6.0 4.4 0.6 −1.0 X

9516 3 girl 4.1 5.8 7.0 4.7 1.2 −1.1

9516 13 girl 8.0 7.9 9.0 7.3 1.1 −0.6

Mean of signed differences (SD) 0.1 (1.6) 0.0 (0.6)Mean of absolute differences (SD) 1.5 (0.6) 0.5 (0.3)

p-value for BXBA being more accurate than ManBA, relative to ReferenceBA P=0.02

# Differences to ReferenceBA larger than 1.5 are highlighted in bold

§ Images where ReferenceBAwas closer to ManBA than to BXBA are marked with “X” BXBAwas significantly closer to ReferenceBA than wasManBA

Fig. 1 Bland-Altman plot of the relation between automatic (BXBA)and the original manual rating (ManBA) in boys (circles) and girls(dots). The difference between the two methods is shown against the

mean of the two methods. The dotted lines indicate the range ofdiscrepancy <1.5 years and the grey line indicated the regression. The 20radiographs that differed by >1.5 years are analyzed in Table 2

1618 Pediatr Radiol (2013) 43:1615–1621

Page 5: Validation of automatic bone age determination in children with congenital adrenal hyperplasia

BXBA. The following precision results were obtained:ManBA: 0.32 years (95%CI: 0.29–0.35); BXBA: 0.21 years(95 % CI: 0.19–0.23). This indicates a significant differencein precision (P<0.001).

Figure 2 shows the longitudinal BXBA and ManBAcurves of three children who had been selected for theirextreme skeletal prematurity as described above (meanbone age advancement: 3.1±3.0 years). None of the 20

images with a bone age discrepancy greater than 1.5between ManBA and BXBA was from any of these 3children. The SD of the signed differences betweenBXBA and ManBA for these three curves was 0.63 years. Acomparison of the smoothness of the longitudinal curvesgenerated by ManBA and BXBA for these 3 children (33triplets spanning less than 1.7 years) yielded precision valuesof 0.24 years (95%CI: 0.17–0.32) forManBA and 0.16 (95%CI: 0.12–0.23) years for BXBA (N.S.).

The difference between automatic and the original manualbone age rating as a function of bone age advancement(Fig. 3) shows a slope of 0.1 years/year (x-intercept = +0.7 years). This means that BoneXpert tends to produceslightly lower ratings than the manual rating with increasingbone age advancement.

Discussion

Rejected images – BoneXpert’s behaviour in the low boneage range

BoneXpert’s inability to rate images below bone age 2 in girlsand 2.5 in boys was a greater limitation in this study than ithad been in other clinical studies on BoneXpert [17, 18]. Incontrast to those studies, the majority of children in thepresent study were regularly monitored for bone age frombirth, which is when CAH is usually diagnosed. Thus, in 111of the 116 images rejected this was due to low bone age, andthe overall rejection rate was greater than 10 %. In a study onshort stature and in another on central precocious puberty, therejection rate was less than 1.5 % (14/1,097 and 9/732,respectively) [17, 18]. Extending BoneXpert’s applicationrange further towards birth may be a worthwhile project inview of the present findings.

Interobserver error between automated and manual ratings

At 0.72 years, the standard deviation of the signed differencesbetween the ratings of BoneXpert and ManBA was in thesame range as in analogous studies on other pathologies [17,18] where standard deviations ranged from 0.71 to 0.8 years.The mean of the absolute differences (0.54 years) comparesfavourably with the levels of interobserver error betweenhuman raters reported in the literature. Berst et al. [20] give0.69±0.48 years for the mean of the absolute differencesbetween two trained observers performing bone age ratingson 107 radiographs in awareness of CA. Expressed in termsof the SD of the signed differences and assuming a normaldistribution, this equates to 0.69/0.8 = 0.86 years. King et al.[21] reported bone age readings of 50 radiographs performedby each of three raters where the SD of the signed differenceswas 0.80 years [20].

Fig. 2 Longitudinal charts of BoneXpert bone age (black) and manual boneage (grey) in 3 children selected according to the following criteria: At least 12images available per child; of these, the 3 children with the most advancedbone age at diagnosis

Pediatr Radiol (2013) 43:1615–1621 1619

Page 6: Validation of automatic bone age determination in children with congenital adrenal hyperplasia

Rerating results: implications for automated and manualrating

In 20 images, the discrepancy between manual and automaticrating was greater than 1.5 years. These were each blindlyrerated by four independent raters. The mean of these fourratings (ReferenceBA) deviated by less than 1.5 years fromBXBA for all images.

We can conclude from our results that the originaldiscrepancies in the 20 rerated images were due more tomanual errors than to errors in automatic rating. This isindicated by 18 reratings being closest to the automatic ratingand 2 closest to the original manual rating, as well as by thesmaller mean absolute difference between BXBA andReferenceBA as compared to that between ManBA andReferenceBA. If ReferenceBA is assumed to represent the“true bone age,” then BXBA was significantly more accuratethan ManBA (P=0.02) in rating these 20 images. This issupplemented by the outcome of our comparison of thesmoothness of longitudinal curves, which clearly showed theautomatic ratingmethod to bemore precise, despite the fact thatthe automated method rated the images independently whereasthe original manual raters usually had the previous rating andthe age of the child available. This tends to enhance smoothnessand improve the precision result derived for manual rating.

Performance of BoneXpert in children with extremelyadvanced bone age

Our comparison of ratings performed on children with faradvanced bone age suggests that this is not a specific sourceof discrepancy between manual and automatic rating in CAH,since the SD of the signed differences between ManBAand BXBA for this subgroup was even smaller – albeitnonsignificantly – than theywere for all children taken together.

Conclusion

BoneXpert supplies satisfactory bone age ratings in childrenwith CAH within its designated bone age application range

(bone age: 2–15 years for girls, 2.5–17 years for boys). Thehigh rate of image rejection found in younger childrenunderscores the need to extend the programme to infants.

Conflicts of interest None.

References

1. White PC, Speiser PW (2000) Congenital adrenal hyperplasia dueto 21-hydroxylase deficiency. Endocr Rev 21:245–291

2. Koletzko B (2007) Kinder-und Jugendmedizin, Heidelberg:Springer Medizin Verlag, Heidelberg

3. Pang SY,Wallace MA, Hofman L et al (1998)Worldwide experiencein newborn screening for classical congenital adrenal hyperplasia dueto 21-hydroxylase deficiency. Pediatrics 81:866–874

4. David M, Sempe M, Blanc M et al (1994) Final height in 69 patientswith congenital adrenal hyperplasia due to 21-hydroxylase deficien-cy. Arch Pediatr 1:363–367

5. Ghali I, David M, David L (1978) Linear growth and pubertaldevelopment in treated congenital adrenal hyperplasia due to 21-hydroxylase deficiency. Obstet Gynecol Surv 33:120–122

6. Jääskeläinen J, Voutilainen R (1997) Growth of patients with 21-hydroxylase deficiency: an analysis of the factors influencing adultheight. Pediatr Res 41:30–33

7. Muirhead S, Sellers EAC, Guyda H (2002) Indicators of adultheight outcome in classical 21-hydroxylase deficiency congenitaladrenal hyperplasia. J Pediatr 141:247–252

8. Van der Kamp HJ, Otten BJ, Buitenweg N et al (2002) Longitudinalanalysis of growth and puberty in 21-hydroxylase deficiencypatients. Arch Dis Child 87:139–144

9. Bonfig W, Bechtold S, Schmidt H et al (2007) Reduced final heightoutcome in congenital adrenal hyperplasia under prednisonetreatment: deceleration of growth velocity during puberty. J ClinEndocrinol Metabol 92:1635–1639

10. Eugster EA, DiMeglio LA,Wright JC et al (2001) Height outcome incongenital adrenal hyperplasia caused by 21-hydroxylase deficiency:a meta-analysis. J Pediatr 138:26–32

11. Hargitai G, Solyom J, Battelino T et al (2000) Growth patterns andfinal height in congenital adrenal hyperplasia due to classical 21-hydroxylase deficiency. Horm Res 55:161–171

12. Lin-Su K, Vogiatzi MG, Marshall I et al (2005) Treatment withgrowth hormone and luteinizing hormone releasing hormoneanalog improves final adult height in children with congenitaladrenal hyperplasia. J Clin Endocrinol Metabol 90:3318–3325

Fig. 3 Difference betweenautomatic (BXBA) and theoriginal manual (ManBA) boneage rating in boys (circles) andgirls (dots) is shown as afunction of bone ageadvancement in CAH

1620 Pediatr Radiol (2013) 43:1615–1621

Page 7: Validation of automatic bone age determination in children with congenital adrenal hyperplasia

13. Martin DD,Neuhof J, Jenni OG et al (2010) Automatic determinationof left-and right-hand bone age in the First Zurich Longitudinal Study.Horm Res Paediatr 74:50–55

14. Thodberg HH, Jenni OG, Caflisch J et al (2009) Prediction of adultheight based on automated determination of bone age. J ClinEndocrinol Metab 94:4868–4874

15. Thodberg HH, Neuhof J, Ranke MB et al (2010) Validation of boneage methods by their ability to predict adult height. Horm ResPaediatr 74:15–22

16. van Rijn RR, Lequin MH, Thodberg HH (2009) Automaticdetermination of Greulich and Pyle bone age in healthy Dutchchildren. Pediatr Radiol 39:591–597

17. Martin DD, Deusch D, Schweizer R et al (2009) Clinical applicationof automated Greulich-Pyle bone age determination in children withshort stature. Pediatr Radiol 39:598–607

18. Martin DD, Meister K, Schweizer R et al (2008) Validation ofautomatic bone age rating in children with precocious and earlypuberty. J Pediatr Endocrinol Metab. 2011;24(11-12):1009–1014

19. Thodberg HH, Kreiborg S, Juul A et al (2009) The BoneXpertmethod for automated determination of skeletal maturity. IEEETrans Med Imaging 28:52–66

20. Berst MJ, Dolan L, Bogdanowicz MM et al (2001) Effect ofknowledge of chronologic age on the variability of pediatric boneage determined using the Greulich and Pyle standards. AJR Am JRoentgonol 176:507–510

21. King DG, Steventon DM, O’Sullivan MP et al (1994) Reproduc-ibility of bone ages when performed by radiology registrars: anaudit of Tanner and Whitehouse II versus Greulich and Pylemethods. Br J Radiol 67:848–851

Pediatr Radiol (2013) 43:1615–1621 1621


Recommended