ava i lab le a t www.sc iencedi rec t .com

journa l homepage : www.e l sev i er . com/ loca te /s iny

Seminars in Fetal & Neonatal Medicine (2007) 12, 332e336

How should we report neonatal outcomes?

Andrew Lyon*

Simpson Centre for Reproductive Health, Royal Infirmary of Edinburgh, Little France,Edinburgh EH16 4SA, Scotland, UK

KEYWORDSNeonatal outcome;Routine data;Neurodevelopmentalassessment;National neonatal dataproject

Summary The aim of neonatal care is to achieve long-term survival free of handicap. Thereis a need for standardised datasets linking perinatal care to outcome at 2 years corrected agefor all babies who have gone through neonatal intensive care. Realistically this can only hap-pen if all the data are collected by the units caring for the babies. This has not been possiblepreviously using routinely collected data because of the poor quality of such information. Re-cent improvements in neonatal data collection along with the development of standardisedneonatal and follow-up datasets make it possible that this could now be achieved from routinedata collected as part of everyday clinical care. It is important that further links with mater-nity and child health systems are developed. The National Neonatal Audit Project, funded bythe Department of Health, will hopefully develop the infrastructure to allow the storage, anal-ysis and rapid reporting of pooled neonatal and follow-up data.ª 2007 Elsevier Ltd. All rights reserved.

Why do follow-up and why collect data?

Providing quality care is a core principle of modern health-care. One definition of quality is doing ‘the right thing, atthe right time, in the right way, for the right person’ andhaving the best possible results.1 This means that health-care providers have a duty to examine their practice and,where appropriate, implement change.

When applied to neonatal care this raises several issues,not least the question of what results we are looking for?There are several time points at which we can study howpractice has affected outcome. Advances in perinatal carehave resulted in improvements in gestation-specific survivalrates such that, in the UK, mortality is no longer a partic-ularly useful measure of quality of care. It is still unclear ifsuch reductions in mortality have been accompanied by any

* Tel.: þ44 0131 242 2567.E-mail address: andrew.lyon@luht.scot.nhs.uk

1744-165X/$ - see front matter ª 2007 Elsevier Ltd. All rights reservedoi:10.1016/j.siny.2007.05.001

changes in the rates of long-term disabilities. If the aim ofour care is to achieve survival free from handicap such thata child will fulfil his maximum potential and lead a healthy,independent adult life, then it is mandatory that anyassessment of quality of care must include the longer-term outcome for these children.

Regular follow-up and assessment are important foridentifying the particular needs of any individual child.This is particularly true for children in high-risk groups,such as those who have needed neonatal intensive care.However, as well as assessing the requirements of anindividual, there is an additional requirement to collectstandardised data on the health status of all children inearly childhood. This can be used to:

� assess the overall health status of all children,� assess the rates of disability in the population in order

to estimate the need for services and to monitor trendsand variations in these rates,

d.

How should we report neonatal outcomes? 333

� monitor these rates in specific high-risk groups (e.g.babies who receive neonatal intensive care), in orderto determine outcome and how it may be altered bychanges in clinical care.

There are two major reasons, therefore, for collectingfollow-up data on all survivors of neonatal intensivecare e to describe the type and incidence of disabilitiesat any particular age and to relate these to aspects of carein the perinatal period. Such data collection will only be ofuse if it can be used to influence outcome by assessing theway we care for these babies. It is essential that anychanges in perinatal management are shown to havea longer-term advantage for the baby needing neonatalintensive care.

When should data be collected?

The optimum time to measure the health status of groupsof children will depend on the reason for the assessment. Itmay take time for the true extent of any particular problemto be evident in the population. In survivors of neonatalintensive care it is important to not only document thelong-term outcome but to try and link this to events withinthe perinatal period. It is generally agreed that 2 years,corrected for gestational age at birth, is the best time forinitial follow-up data to be collected, since2:

� at this age the findings are still likely to be relevant tocurrent clinical practice and there are unlikely to bemajor changes within a 2-year period that significantlyinfluence survival and outcome,� the effects of social and demographic factors will be

minimised,� at this age it should still be relatively easy to achieve

a good follow-up rate.

There is a need to follow specific groups of children wellbeyond 2 years corrected age.3 Such assessments may wellreflect performance of the child more accurately and thismay be important in helping plan services, such as educa-tion, for these children. However, the findings will haveless relevance to current perinatal care and the greatereffects from social and demographic factors will make itmore difficult to relate outcome to any treatment or prob-lem in the early neonatal period. With increasing age thereis also likely to be a greater ‘fall out rate’ with morechildren being lost to follow up.

How can the data be collected?

Linking aspects of perinatal care to longer-term outcomerequire the collection of relevant data during the neonatalperiod and at follow-up at 2 years corrected age. Theusefulness of such data will depend on its completeness andquality.

Formal studies with independent funding are likely toproduce the best quality data but are often only addressingvery specific questions. Retrospective case-controlled stud-ies have been used but the randomised controlled trial(RCT) is the ‘gold standard’ for measuring the efficacy of

any new therapeutic intervention in perinatal care. TheRCT is, however, an expensive option with funding for suchtrials becoming increasingly difficult to obtain. If webelieve that the aim of neonatal care is to achieve survivalfree of handicap then it is important that all trials considernot just short term but also longer-term effects of thetreatment under study.

Following specific cohorts of babies can give a ‘snapshot’picture at one period in time. One of the best examples ofthis has been the EPICure study, which followed, frombirth, all babies born at 22e25 weeks’ gestation in the UKbetween March and December 1995.4,5 Complete and veryaccurate data were collected but even this relatively smallgroup of babies involved an enormous amount of work andexpense. There is already criticism that the original EPICuredata no longer reflects current perinatal care. The study isnow being repeated with a new cohort of babies, this timeincluding those of 26 weeks’ gestation. This larger group ofbabies has resulted in a pragmatic decision not to includedeliveries from the whole of the UK. This reflects thetremendous amount of work required to collect such data.

The role of registers in the planning and provision ofhealthcare for children and health services research hasbeen much debated.6 A register is a list of children witha particular predefined attribute, usually a disease or a con-dition. They can also contain information on children whoare thought to be high risk. They have standardised defini-tions for inclusion and exclusion that should allow thecombination of data from registers in several differentcountries.

Registers are a valuable resource but are under threatbecause of concerns about data protection and the majorresource implications in making sure that all children arereported and the diagnoses are accurate.

The birthweight-specific rate of cerebral palsy (CP) isnot an accurate reflection of the quality of perinatal care,and trends observed on CP registers need to be interpretedwith care.7 However, a recent report combining data fromseveral European CP registers, including the UK, has shownthat the prevalence of CP in children with a birthweight ofless than 1500 g has fallen between 1980 and 1996.8 Theprevalence of CP in other groups has remained the same.These data are from a very large population base and giveconvincing evidence that infants with a birthweight ofless than 1500 g and, in particular, those less than 1000 g,now have a better chance of survival without severe neuro-logical impairment. This has important implications forparents, health services and society, but such registerscannot link improvement in outcome directly to anyspecific changes in perinatal care. Also, the results reflecta period over a decade ago and hence may not be immedi-ately applicable to current perinatal practices.

The RCT addresses very specific questions, while regis-ters could be further developed for specific problems andcohort studies can give accurate snapshots for selectedgroups of babies. All of these are expensive options becausethey have to be set up from scratch each time. There canbe quite a delay between the study and the final reportmaking it difficult to relate the findings to current clinicalpractice. These methods are unlikely to give us a means ofcontinually monitoring over time the outcome for all babieswho have needed neonatal care.

334 A. Lyon

What is needed is a method of ongoing surveillance ofoutcome that:

� is easy to apply to all babies who need neonatal inten-sive care,� is relatively cheap,� can link outcome to current perinatal care,� can report rapidly so that findings are relevant to cur-

rent methods of care.

Such a surveillance system will help in the assessment ofthe outcome of the ‘whole package’ of perinatal care,much of which has not been subjected to the rigour ofa randomised trial. It could also offer a means of bench-marking units, as well as monitoring them to make surethey are achieving any defined outcome standards.

Can anything useful be obtained from current childhealth surveillance systems? There have been severalreviews looking at whether data that are collected rou-tinely on these systems can be used to monitor the long-term outcome of babies who have been through intensivecare. The overall opinion has been very negative, the databeing incomplete, inaccurate and not in a standardisedformat that can easily be linked to other databases.9,10 De-spite reports over many years stressing the need to developa more ‘joined up’ approach to data collection,11 there isno evidence that current child health surveillance systemsare of use for this purpose. The NHS IT project, Connectingfor Health, is an ambitious attempt to bring health com-puter systems together. It is unlikely that this projectwill, in the foreseeable future, improve the quality ofoutcome data for high-risk groups of children, let alonelink this with meaningful perinatal data. It is important atthis stage that we, the professionals, continue to lead theway in determining what datasets need to be collected toassess and monitor outcome for babies and to link this toaspects of perinatal care.

One of the required standards, published by the BritishAssociation of Perinatal Medicine (BAPM), for hospitalsinvolved in neonatal intensive and high dependency careis that they collect data to measure their workloads andshort-term outcomes.12 There is also a requirement thatunits develop and run a system for follow-up of all babieswho have been under their care. Most neonatal units arecollecting data, with the majority doing this electronically.It would seem sensible to explore whether current systemsof data collection used by neonatal units could be used tomonitor any link between perinatal care and outcome.

Units collect data for their own local review and audit ofoutcome. Pooling data would allow comparison of outcomesbetween similar units as well as giving larger numbers fora more meaningful analysis of the links between perinatalcare and outcome. Pooled population data can be used forcomparison of outcomes with other countries and health-care systems. Comparing the practice in units with widelydifferent outcomes can be used in quality improvementexercises that aim to raise the standards of those perform-ing poorly.

Data routinely collected by neonatal units may wellshow associations between perinatal care and outcome butwill be unlikely to be able to prove any definite causal links.However, the collection and analysis of such data may show

areas of care that would benefit from inclusion in a moreformal RCT.

To be able to pool data from several sources there mustbe agreement on what items need to be collected tomeasure outcome. It is also essential that all must usethe same definition for each of the data items. Even slightdifferences in the definitions used can make it impossible tocombine and compare data from several units. This is oneof the greatest challenges to the use of routinely collecteddata for benchmarking and population studies.

Comparison of data will only be meaningful if the samedenominators are used. There are many reasons whyapparent differences in outcome for liveborn babies maynot be a reflection of quality of perinatal care.13 Case mixand disease severity are important and there must bestandardisation of all data for these factors. Differencesin the management and registration of babies at themargins of viability can have significant effects on gesta-tion-specific outcome measures. The failure to includebabies dying before admission to a neonatal unit will havesimilar effects. Differing rates of early termination forcongenital malformations and intrapartum stillbirths willaffect the number of live births and could be the reasonfor differences in outcome of those born alive, ratherthan anything to do with quality of postnatal care.

Such problems in the use of routinely collected data arenot insurmountable but there is a need for standardisationif anything is to be done with such data beyond its use ata local level.

Current position of routine data collection inthe UK

There are two datasets to consider if we are to link longer-term outcome to perinatal care: one which describes thehealth status of the baby at follow-up and the other whichincludes relevant perinatal data. How units will actuallycollect the data needs consideration. There also must be inplace an infrastructure that allows storage, analysis andrapid reporting of the pooled data.

The datasets were considered in the report of twoworking groups convened by the National Perinatal Epide-miology Unit (NPEU) and Oxford Regional Health Authority(1994).2

Perinatal data

The small perinatal dataset from the NPEU working groupsincludes the major predictors of outcome, i.e. gestation andbirthweight. The British Association of Perinatal Medicine(BAPM) has published a neonatal dataset so that units canprepare a standardised annual report.14 This datasetincorporates measures of workload as well as clinical careand forms the basis of much of the current neonatal data col-lection throughout the UK. BAPM is involved in work to en-sure the standardisation of definitions of all data itemsbeing collected by the neonatal systems in use across the UK.

Data collection has resource implications wherever ithappens. Routine data are collected by most units, but thecompleteness and quality are very variable and oftendependent on how seriously the unit views data collection

How should we report neonatal outcomes? 335

as part of clinical care. The Neonatal Survey is an ongoingstudy of neonatal intensive care activity that started in theformer Trent Health Region and has since expanded toother areas. This survey employs its own staff to visit eachunit and extract the data needed from clinical notes. Thisensures complete, good quality data but would be anexpensive model to roll out throughout the UK. In theabsence of independent staff visiting, we rely on unitsgathering their own data. This will only be reliable whendatasets are simple and all items are collected as a routinepart of clinical care. Improvements in software can helpwith data checking and validation but no data item shouldhave to be collected more than once.

Web-based systems running on the NHS net are beingdeveloped and are used across much of the UK for neonataldata collection. Examples include the following:

� The Scottish Birth Record,� MANNERS (Maternity and NeoNatal Electronic Record

System) used predominantly in the West Midlands,� The Neonatal.net platform, which supports an increas-

ing number of neonatal data collection projects. Theseinclude SEND (South England Neonatal Data Collectionbut extending to other parts of the UK) and, in Ireland,NICORE (Neonatal Intensive Care Outcome Researchand Evaluation).

The development of such systems is improving thequality of routinely collected neonatal data. It is easier tomaintain standard datasets as well as allowing the additionor removal of data items to reflect changes in perinatalpractice.

The introduction of mandatory datasets linked to fund-ing of service would improve the quality of routine datacollection. Within England and Wales the Neonatal CriticalCare Minimum Dataset, to support the introduction ofPayment by Results, will be mandatory from April 2007.At present this only includes data items used to define levelof care as a means of assessing payment. However,hospitals are going to have to find the resources to collectthis neonatal dataset, which may lead to a greater accep-tance of the importance of routine data collection.

Outcome data

If the aim is to collect follow-up data, at 2 years correctedage, on all children who have been through neonatalintensive care then this will have to be collected locally,again as part of routine follow-up. What dataset is col-lected will depend on the purpose of the follow-up. It islikely that research projects will require more detailedinformation than that needed for a surveillance pro-gramme. If follow-up is being done by clinical staff thenthe dataset must be easy to collect, avoiding complex testsof neurodevelopment for which training is required.

The NPEU working group developed the Health StatusQuestionnaire as a method of routine assessment for allchildren at 2 years of age.2 It is rapid to use and requires noformal training. Neither this questionnaire nor routinelycollected data were effective in providing complete fol-low-up data on children who were born preterm.15 With

some modifications it has been favourably compared withestablished methods of assessing disability.16

Although easy to use, and potentially effective, theHealth Status Questionnaire has, over the years, tended tobe modified locally to meet the needs of individual units. Toallow data to be pooled what we need is an agreed,standardised dataset. BAPM is currently working to producea dataset that can be used for the surveillance of outcomeat 2 years corrected age.

Data linkage, pooling and analysis

Datasets need to be linked together and this requiresa unique identifier such as the NHS number (or CHI numberin Scotland). Although neonatal data collection has ex-panded rapidly across the UK, the present links withmaternity and child health systems are limited. There iswork underway to define a national maternity dataset and itis important that this will ultimately link to a standardisedneonatal dataset. There is no reason why the quality of dataheld on child health surveillance systems should not beimproved and standardised, making it possible to use theNHS number to link perinatal data to outcome measuresbeyond 2 years of age. There are many issues around dataconfidentiality that need to be addressed, but it would besad if these prevented the use of data to the benefit of thishigh-risk group of children. It could be argued that it wouldbe unethical to collect such data and not use it in a con-structive way. At the very least there must be ways found toreliably anonymise the data, although an identifier such asthe NHS number will always be needed for data linkage.

The current systems mentioned above (SEND, MANNERS,NICORE, Scottish Birth Record, Neonatal Survey, etc.) arealready analysing and reporting pooled data from unitswithin their specific areas. The National Neonatal AuditProject (NNAP) has been funded by the Department ofHealth through the Healthcare Commission and is beingimplemented across England. This project aims to collecta small neonatal dataset from all units. Ultimately this willinclude 2-year follow-up data and NNAP is working withBAPM in the development of this dataset. NNAP is pro-fessionally led through the Royal College of Paediatrics andChild Health. It has a remit to collect, analyse and report onpooled neonatal data and will hopefully develop theinfrastructure needed for ongoing surveillance of outcomelinked to perinatal care.

With the advances in neonatal data collection, the useof NHS web-based systems and the acceptance of stand-ardised datasets it is possible to see how we may, in the UKin the not too distant future, be confident in the use ofroutinely collected data to link 2-year outcome to aspectsof care within a neonatal intensive care unit. It will takelonger to develop links to maternity systems and to sub-sequent child health data. In 1999, Field et al. discussedthe difficulties of using routine data for benchmarking.17

The requirements highlighted in their review have notchanged and can be summarised as follows:

� Data collection must become a core funded aspect ofclinical care, not an optional extra. Extra costs can bejustified by the benefits that will follow in terms of

336 A. Lyon

understanding the process of care, what works andwhat does not.� There should be a mandatory national perinatal dataset

that is extremely simple and should include the NHSnumber. This is to be collected for all infants<32 weeks’ gestation and all those who get intensivecare as well as recognised priority groups.� These same children should have their health status

ascertained at 2 years using a simple structured ques-tionnaire carried out by health personnel based in thecommunity of the parents.� There is a clear role for public health doctors within

health authorities, in relation to many aspects of childhealth, to receive and review mandatory perinataldata. (The development of a professionally led dataanalysis unit may be another way forward to makesure these data are analysed and reportedappropriately).� Linkage of anonymised pooled health data should be

exempted from data protection regulation.

We have gone some way to achieving these require-ments, but further work is needed if we are to have a meansof routine surveillance of outcome linked to perinatal care.

References

1. Agency for Healthcare Research and Quality. Your Guide tochoosing quality health care: a quick look at quality. Availablefrom: <http://www.ahrq.gov/consumer/qnt/qntqlook.htm>[accessed Jan 2007].

2. National Perinatal Epidemiology Unit and Oxford RegionalHealth Authority. Disability and perinatal care: measurementof health status at two years. Report of two working groups.Oxford: National Perinatal Epidemiology Unit; 1994.

3. Marlow N. Neurocognitive outcome after very preterm birth.Arch Dis Child Fetal Neonatal Ed 2004;89:F224e8.

4. Wood NS, Marlow N, Costeloe K, Gibson AT, Wilkinson AR.Neurologic and developmental disability after extremely

preterm birth. EPICure Study Group. N Engl J Med 2000;343:378e84.

5. Costeloe K, Hennessy E, Gibson AT, Marlow N, Wilkinson AR. TheEPICure study: outcomes to discharge from hospital for infantsborn at the threshold of viability. Pediatrics 2000;106:659e71.

6. Johnson A. Use of registers in child health. Arch Dis Child 1995;72:474e7.

7. Pharaoh P, Cooke T, Cooke RW, Rosenbloom L. Birthweight spe-cific trends in cerebral palsy. Arch Dis Child 1990;65:602e6.

8. Platt MJ, Cans C, Johnson A, et al. Trends in cerebral palsyamong infants of very low birthweight (<1500 g) or born pre-maturely (<32 weeks) in 16 European centres: a databasestudy. Lancet 2007;369:43e50.

9. Dawson D, Perkins M, Draper E, Johnson A, Field D. Are outcomedata regarding the survivors of neonatal care available from rou-tine sources? ArchDisChildFetalNeonatalEd1997;77:F206e10.

10. Johnson A, King R. Can routine information systems be used tomonitor serious disability? Arch Dis Child 1999;80:63e6.

11. Wanless D. Securing good health for the whole population.London: HM Treasury; 2004.

12. BritishAssociationofPerinatalMedicine. Standards forhospitalsproviding intensive and high dependency neonatal care. 2nd ed.London: BAPM. Available from: <http://www.bapm.org/media/documents/publications/hosp_standards.pdf>; 2001.

13. Draper ES, Zeitlin J, Field DJ, Manktelow BN, Truffert P. Mor-tality patterns of very preterm babies: a comparative analysisof two European regions in France and England. Arch Dis ChildFetal Neonatal Ed; 2007 Jan 9 [Epub ahead of print].

14. BritishAssociation of PerinatalMedicine.BAPM neonatal datasetfor the annual reporting of data by neonatal intensive careunits. London: BAPM. Available from:<http://www.bapm.org/media/documents/publications/dataset_fullreview_20040300.pdf>; 2004.

15. Field D, Draper ES, Gompels MJ, et al. Measuring later healthstatus of high risk infants: randomised comparison of two sim-ple methods of data collection. Br Med J 2001;323:1e5.

16. Jones HP, Guildea ZES, Stewart JH, Cartlidge PHT. The healthstatus questionnaire: achieving concordance with publisheddisability criteria. Arch Dis Child 2002;68:15e20.

17. Field D, Manktelow B, Draper ES. Bench marking and perfor-mance management in neonatal care: easier said than done!.Arch Dis Child Fetal Neonatal Ed 2002;87:F163e4.