Diagnostic accuracy of PAT-POPS and ManChEWSfor admissions of children from the emergencydepartmentSarah Cotterill,1 Andrew G Rowland,2,3 Jacqueline Kelly,2 Helen Lees,2

Mohammed Kamara2

1Centre for Biostatistics,University of Manchester,Manchester, UK2Emergency Department, NorthManchester General Hospital,Manchester, UK3The University of Salford,Salford, UK

Correspondence toDr Sarah Cotterill,Research Fellow, Centre forBiostatistics, Jean McFarlaneBuilding, University ofManchester, Oxford Road,Manchester, M13 9PL, UK;sarah.cotterill@manchester.ac.uk

Received 12 January 2015Revised 12 February 2016Accepted 14 March 2016Published Online First11 April 2016

▸ http://dx.doi.org/10.1136/emermed-2016-205877

To cite: Cotterill S,Rowland AG, Kelly J, et al.Emerg Med J 2016;33:756–762.

ABSTRACTBackground The Pennine Acute Trust (PAT) PaediatricObservation Priority Score (PAT-POPS) is a specificemergency department (ED) physiological andobservational aggregate scoring system, with scores of0–18. A higher score indicates greater likelihood ofadmission. The Manchester Children’s Early WarningSystem (ManChEWS) assesses six physiologicalobservations to create a trigger score, classified asGreen, Amber or Red.Methods Prospectively collected data were used tocalculate PAT-POPS and ManChEWS on 2068 patientsaged under 16 years (mean 5.6 years, SD 4.6)presenting over 1 month to a UK District GeneralHospital Paediatric ED. Receiver operating characteristics(ROC) comparison, using STATA V.13, was used toinvestigate the ability of ManChEWS and PAT-POPS topredict admission to hospital within 72 h of presentationto the ED.Results Comparison of the area under the ROC curveindicates that the ManChEWS ROC is 0.67 (95% CI0.64 to 0.70) and the PAT-POPS ROC is 0.72 (95% CI0.68 to 0.75). The difference is statistically significant. Ata PAT-POPS cut-off of ≥2, 80% of patients had theiradmission risk correctly classified (positive likelihoodratio 3.40, 95% CI 2.90 to 3.98) whereas forManChEWS with a cut off of ≥Amber only 71% ofpatients were correctly classified (positive likelihood ratio2.18, 95% CI 1.94 to 2.45).Conclusions PAT-POPS is a more accurate predictor ofadmission risk than ManChEWS. Replacing ManChEWSwith PAT-POPS would appear to be clinically appropriatein a paediatric ED. This needs validation in a multicentrestudy.

INTRODUCTIONHealth professionals make judgements on whetherchildren attending emergency departments (EDs)require hospitalisation, or can safely be sent home.These judgements require a complex assessment ofthe child’s health and an estimation of the potentialfor improvement or deterioration. Since at least2006 it has been recommended that early identifica-tion systems to recognise children developing criticalillness should be used.1 Many paediatric earlywarning scores (EWS) use track and trigger systems,relying on repeated observations over time,intended for use with hospitalised children,2–9 topredict which children are likely to deteriorate,rather than who requires admission from an ED.There is a need for a specific ED early warning

system, validated on ED patients.10 In the absenceof an ED EWS, our ED used The Royal ManchesterChildren’s Hospital Early Warning System(ManChEWS), a track and trigger system.11

Variation in six physiological parameters (RR,oxygen requirement, heart rate, systolic BP, capillaryrefill time and conscious level) is scored duringroutine nursing observations (figure 1). WhileManChEWS correctly identifies the deterioratingchild it overtriggers which may lead staff to becomeimmune to the score.4

The Pennine Acute Trust (PAT) PaediatricObservation Priority Score (PAT-POPS) is a newEWS, designed for use in children’s ED, combiningphysiological measurement and clinical observationinto an aggregate scoring system (figure 2). Thetarget user of PAT-POPS is nursing staff in the EDas part of their post-triage initial assessment.PAT-POPS has been adapted from the PaediatricObservational Priority Score (POPS),12 13 designed

Key messages

What is already known on this subject?▸ There is a need for a specific emergency

department early warning system for children.▸ Manchester Children’s Early Warning System

(ManChEWS) is one scoring system, whichcorrectly identifies the deteriorating child, butcan overtrigger which may lead staff to becomeimmune to the score.

▸ Paediatric Observation Priority Score (POPS) is anew scoring system, combining physiologicalmeasurement and clinical observation.

What might this study add?▸ A Pennine Acute Trust (PAT) Paediatric

Observation Priority Score (PAT-POPS) score ≥2would correctly identify about 50% of childrenwho need to be admitted from the ED, and aPAT-POPS score <2 would correctly identify85% of cases who could be discharged fromthe ED.

▸ PAT-POPS has slightly higher diagnosticaccuracy for predicting the likelihood ofadmission than an existing tool often used inthis population, ManChEWS.

▸ PAT-POPS can be used for patients with eithertrauma or illness.

756 Cotterill S, et al. Emerg Med J 2016;33:756–762. doi:10.1136/emermed-2015-204647

Original article on A

ugust 31, 2020 by guest. Protected by copyright.

http://emj.bm

j.com/

Em

erg Med J: first published as 10.1136/em

ermed-2015-204647 on 11 A

pril 2016. Dow

nloaded from

by the Paediatric Emergency Medicine Leicester AcademicGroup.

The original POPS demonstrated utility as a patient safetysystem in an analysis of the admission length of 24 000 patientswho attended a UK ED.14 With an area under the curve (AUC)for the receiver operating characteristic (ROC) curve of 0.8, itwas concluded that POPS may assist in reducing unnecessaryadmissions and preventing episodes of missed or incorrectdiagnoses.

Over a period of 1 month, we calculated PAT-POPS andManChEWS scores for 2068 children attending our paediat-ric ED, and recorded whether the child was admitted or dis-charged. We compared the diagnostic accuracy of the twoscoring methods in estimating the likelihood of admission ordischarge. The overall aim of this study was to compare thediagnostic accuracy of PAT-POPS and ManChEWS for esti-mating whether or not hospital admission is required amongchildren presenting to an ED with trauma or medicaldiagnoses.

METHODParticipantsThe study population was children aged under 16 years attend-ing the ED of North Manchester General Hospital (part ofPAT) during March 2012. Children who left the ED before theycould be assessed for admission, or where insufficient data wereavailable to calculate PAT-POPS and ManChEWS, wereexcluded.

Test methodsThe reference standard for our study was admission to hospitalwithin 72 h of first presentation at the ED. This was a prospect-ive study of a consecutive series of patients: data collection wasplanned before the index tests and reference standard were per-formed. The decision on whether or not to admit a child toinpatient care was made by the clinician seeing the patient,using their subjective clinical experience as well as departmentalguidelines, including ManChEWS. The disposal outcome (dis-charged or admitted) for each attendance was recorded on the

Figure 1 Manchester Children’s EarlyWarning System (ManChEWS) scoringsystem.

Cotterill S, et al. Emerg Med J 2016;33:756–762. doi:10.1136/emermed-2015-204647 757

Original article on A

ugust 31, 2020 by guest. Protected by copyright.

http://emj.bm

j.com/

Em

erg Med J: first published as 10.1136/em

ermed-2015-204647 on 11 A

pril 2016. Dow

nloaded from

electronic patient record. Information was collected onre-presentations, and where a child re-presented to the ED withthe same clinical problem within 72 h and was admitted, wecounted that as an admission for the original presentation. Dataon admission were recorded by hospital staff in the hospitalelectronic record, and then extracted by the research teamretrospectively.

ManChEWS is scored Green, Amber or Red.11 The scoreencourages observation monitoring and empowers nursing staffby providing clear instructions of what actions should be takendependent on the status of the child. All observations must bewithin the normal range for the age of the child for the awardof Green status. Any physiological parameters that are abnor-mal, but within the defined range, lead to Amber status. Anyparameters that are very abnormal and which lie outside of theGreen or Amber ranges result in a Red status, indicating that thechild has potentially significant physiological disturbance.

PAT-POPS is assessed as a score between 0 (likely low risk ofserious illness) and 18 (likely high risk of serious illness) and is achecklist which quickly scores acutely ill children on age-relatedphysiological measures (heart rate, RR, temperature) and behav-ioural and risk-identifiers (such as oxygen saturations, breathingpattern, conscious level, nurse’s judgement of how well thechild is, child’s behaviour) using easy to collect data (figure 2).Measurements of the physiological variables and subjectiveassessments necessary to calculate ManChEWS and PAT-POPSfor each patient were taken by nursing staff in the ED either atthe point of triage or during the child’s assessment in the ED.

Before the study, nursing staff were trained in the use ofPAT-POPS. Nursing staff were already familiar withManChEWS. Patient data for the PAT-POPS and ManChEWSassessment were collected prior to the admission decision, sothere was blinding to the outcome. The persons who transcribedthe data from the electronic patient record to the PAT-POPS

data sheet did so retrospectively, some time after admission, andso were not blinded to the outcome. The doctor making theadmission decision was not blinded to the ManChEWS scoreand may have used it in their decision-making. Doctors wereblinded to the PAT-POPS scores, which were recorded onto datacapture sheets rather than directly into the patient’s notes. Thedoctors received no training on PAT-POPS to minimise thechances of them using it as part of a decision-making process ifthey happened to see the data capture sheet.

Statistical methodsWe report the age, gender and diagnosis of our sample ofpatients, using descriptive statistics. We have calculated the sensi-tivity and specificity of PAT-POPS and ManChEWS to predictadmission and we present this data as comparative ROC curves.We report the positive and negative likelihood ratios at differentcut points of PAT-POPS and ManChEWS.15 We present 95%CIs and p values, as appropriate. We have compared the sensi-tivity and specificity of PAT-POPS to predict admission for sep-arate groups of children with illness or trauma, using ROCanalysis, and followed the DeLong method to compare the AUCof ROC curves for PAT-POPS and ManChEWS.16 The datawere entered into Microsoft Excel and analysed using STATAV.13 (StataCorp. 2013. Stata Statistical Software: Release 13.College Station, TX: StataCorp LP).

EthicsThis service evaluation was approved by the hospital’s auditdepartment and no formal ethical approval was required as noclinical decisions were made on the basis of the PAT-POPS dataand we continued to manage each patient in accordance withour ED guidelines.

Figure 2 North Manchester modified Pennine Acute Trust (PAT) Paediatric Observation Priority Score (PAT-POPS) chart.

758 Cotterill S, et al. Emerg Med J 2016;33:756–762. doi:10.1136/emermed-2015-204647

Original article on A

ugust 31, 2020 by guest. Protected by copyright.

http://emj.bm

j.com/

Em

erg Med J: first published as 10.1136/em

ermed-2015-204647 on 11 A

pril 2016. Dow

nloaded from

RESULTSParticipantsBetween 1 March 2012 and 31 March 2012, 2539 childrenattended the paediatric ED. We have omitted 87 children withno reference standard who left the ED without being seen, and384 children with no PAT-POPS recorded. There are two likelyreasons for not recording PAT-POPS. First, where there was nowait in the ED a disposal outcome decision may have beenmade before nursing assessment. Second, PAT-POPS was a newtool and some staff may have forgotten to calculate a PAT-POPSfor each patient. The analysis is based on the remaining 2068patients (figure 3). Among the 2068 children, the mean age was5.6 years, median 4.1 years (range 1 day to 15 years), 954(46%) presented with a trauma and 1114 (54%) with a medicalcondition.17 A comparison of children with and without aPAT-POPS recorded score indicates that the children who wereassessed using PAT-POPS were, on average, younger (5.6 yearscompared with 8.2 years), and were less likely to attend withtrauma (46.1% compared with 53.6%), more likely to re-attendwithin 72 h (3.7% compared with 1.6%) and more likely to beadmitted (15.8% compared with 4.4%). All these differencesare statistically significant (p<0.05), using Mann–Whitney orproportion tests, as appropriate. It appears that the 384 childrenwho were not assessed for PAT-POPS were less ill than thosewho were assessed.

Hospital admissionOf the 2068 children, 317 (15.3%) were admitted to hospital atfirst attendance. Seventy-six children (3.7%) re-attended the ED

with the same complaint one or more times within 72 h of firstattendance and 15 (19.7%) of these were admitted (3 of whomhad also been admitted on first attendance, and then dis-charged). In total, 327 (15.81%) were admitted to hospitalwithin 72 h of first attendance.

Test resultsMeasurement of the variables necessary to calculateManChEWS and PAT-POPS was undertaken as soon as possibleafter arrival to the ED. Over 95% of patients were seen andeither admitted or discharged within a period of 4 h from thetime of arrival.

ManChEWS resultsAcross the 2068 patients, 1397 (67.55%) were Green, 573(27.71%) Amber and 98 (4.74%) Red (table 1).

PAT-POPS resultsThe mean PAT-POPS score across all 2068 patients was 0.9 (SD1.58). The median was 0. Overall 62% of patients scored aPAT-POPS of 0. No patient had a PAT-POPS score of greaterthan 12 (table 1).

No adverse events were reported from performing any of theindex tests or reference standard. There were no deaths of chil-dren in the ED during the study period.

Diagnostic accuracy of ManChEWS and PAT-POPSSensitivity and specificitySensitivity and specificity of ManChEWS to predict admissionwere calculated at each ManChEWS value (table 2). A Red

Figure 3 Study flow diagram.

Cotterill S, et al. Emerg Med J 2016;33:756–762. doi:10.1136/emermed-2015-204647 759

Original article on A

ugust 31, 2020 by guest. Protected by copyright.

http://emj.bm

j.com/

Em

erg Med J: first published as 10.1136/em

ermed-2015-204647 on 11 A

pril 2016. Dow

nloaded from

ManChEWS score identifies around 16% of those who wereadmitted, and successfully identifies almost all of those whowere not admitted. An Amber or Red score identifies almost60% of those who were admitted, and 73% of those who weredischarged from the ED. Looking at the positive likelihoodratios, children with a ManChEWS score of Amber or Red aretwice as likely to be admitted as children assessed as Green.Children with a ManChEWS score of Red are almost six timesas likely to be admitted, compared with children assessed asGreen or Amber.

Sensitivity and specificity of PAT-POPS to predict admissionwere calculated at each PAT-POPS value (table 2). A PAT-POPSscore of 9 or above correctly identifies all of those who weresent home, but has poor sensitivity. Sensitivity improves as thecut point lowers, without great loss of specificity until under2. A PAT-POPS score of 2 or more successfully identifies 50%

of those who were admitted, and 85% of those who were senthome. Looking at the positive likelihood ratios, children with aPAT-POPS cut point of 2 or more are more than three times aslikely to be admitted as children with 0 or 1.

Receiver operating characteristicsSensitivity was plotted against (1−specificity) for each pos-sible cut-off of ManChEWS and PAT-POPS in ROC curves(figure 4). The area under the ROC curve for ManChEWSis 0.67 (95% CI 0.64 to 0.70) and for PAT-POPS is 0.72(95% CI 0.68 to 0.75). The difference is statistically signifi-cant (p<0.01).

Use of PAT-POPS for trauma and medical patientsSensitivity was plotted against (1−specificity) to draw separateROC curves for the use of PAT-POPS among medical andtrauma patients (figure not shown). The area under the ROCcurve is 0.73 for medical patients and 0.69 for trauma patients.This analysis suggests that the usefulness of PAT-POPS topredict admission is similar among both trauma and medicalpatients, although slightly higher in medical patients. Thepattern is repeated for ManChEWS: in medical patients, thearea under the ROC curve is 0.69, and in trauma patients it is0.66.

LIMITATIONSThis is a single-centre study and the results could have beendifferent in another centre with an alternative arrangement ofservices and a different admission threshold. The outcomemeasure used in the study is whether the child was admittedto hospital, as assessed by an ED clinician. We endeavouredto increase the robustness of that measure by including anyreadmissions within 72 h. Future studies could consider levelof inpatient care, admissions to other hospitals and length ofstay.

Decision-making on a heterogeneous population of medicaland trauma patients differs widely. There are some conditionswhich automatically trigger an admission regardless of thePAT-POPS (such as deliberate self-harm, child protection casesor a child with a fracture requiring operative management).

This study was based on patients who attended the ED during1 month (March) and diagnoses in paediatrics do have a sea-sonal variation, with higher rates of respiratory conditions inwinter and minor trauma in summer. March was chosenbecause this month coincided with some dedicated time madeavailable in work schedules for members of the study team. Infuture studies sampling patients from throughout the year willbe important.

Some patients who attended the ED during the month of thestudy did not have PAT-POPS recorded, because of an absenceof some of the data used to calculate the score. The missingdata could not be calculated retrospectively because PAT-POPSincludes subjective nurse assessments of the child’s behaviourand condition, which is not routinely collected.

Although our results show that PAT-POPS appears slightlybetter than ManChEWS at predicting admission of childrenfrom the ED, PAT-POPS scores are disadvantaged by nursesbeing less familiar with the PAT-POPS tool and hence morelikely to make errors in scoring and clinicians not being blindedagainst the ManChEWS score.

Nonetheless, it is important to acknowledge that the differ-ences between those two scoring systems may not be clinicallysignificant.

Table 1 A cross tabulation of Manchester Children’s EarlyWarning System (ManChEWS) and Paediatric Observation PriorityScore (POPS) scores by admission

Admitted Discharged Total

ManChEWS scoreGreen 132 1265 1397

9.45% 90.55% 100%Amber 144 429 573

25.13% 74.87% 100%Red 51 47 98

52.04% 47.96% 100%

PAT-POPS score0 108 1164 1272

33.0% 66.9% 61.5%1 57 323 380

17.4% 18.6% 18.4%2 44 137 181

13.5% 7.9% 8.8%3 42 56 98

12.8% 3.2% 4.7%4 18 32 50

5.5% 1.8% 2.4%5 18 20 38

5.5% 1.2% 1.8%6 10 6 16

3.1% 0.3% 0.8%7 10 0 10

3.1% 0.0% 0.5%8 9 3 12

2.8% 0.2% 0.6%9 4 0 4

1.2% 0.0% 0.2%10 3 0 3

0.9% 0.0% 0.2%11 1 0 1

0.3% 0.0% 0.1%12 3 0 3

0.9% 0.0% 0.2%Mean PAT-POPS 2.3 0.6 0.9SD 2.62 1.11 1.58

Total, N 327 1741 2068

PAT-POPS, Pennine Acute Trust (PAT) Paediatric Observation Priority Score.

760 Cotterill S, et al. Emerg Med J 2016;33:756–762. doi:10.1136/emermed-2015-204647

Original article on A

ugust 31, 2020 by guest. Protected by copyright.

http://emj.bm

j.com/

Em

erg Med J: first published as 10.1136/em

ermed-2015-204647 on 11 A

pril 2016. Dow

nloaded from

DISCUSSIONNo universally validated children’s EWS exists to predict likeli-hood of admission or discharge from the ED with illness orinjury.

This research has demonstrated that, among children aged 0–16 years, PAT-POPS has slightly higher diagnostic accuracy forpredicting the likelihood of admission than an existing tooloften used in this population, ManChEWS, and that it can beused for patients with either trauma or illness.

The use of PAT-POPS in trauma patients as well as medicalpatients may sound counterintuitive: the decision to admit apatient with a trauma diagnosis from the ED can be based onfactors and assessments that are quite different from thosepatients with a medical diagnosis (such as the need for operativeintervention). Even though the original POPS was intended tobe used on medical rather than trauma patients, the authors ofthis study believed that it would be interesting to evaluate howwell PAT-POPS functioned on patients with a trauma diagnosisas well as a medical diagnosis. Some trauma patients would

Table 2 Sensitivity and specificity of Manchester Children’s Early Warning System (ManChEWS) and Paediatric Observation Priority Score(POPS) to predict admission to the emergency department (ED)

Cut point for admission decision Sensitivity Specificity Likelihood ratio+ Likelihood ratio−

ManChEWSAdmit all (Green, Amber, Red) 100.00% 0.00% 1.00Admit if Amber or Red 59.63% 72.66% 2.18 0.56

(54.08 to 64.96) (70.49 to 74.73) (1.94 to 2.45) (0.49 to 0.64)Admit if Red 15.60% 97.30% 5.78 0.87

(11.93 to 20.09) (0.96 to 0.98) (3.96 to 8.43) (0.83 to 0.91)Admit none 0.00% 100.00% 1.00

POPS(≥0) 100.00% 0.00% 1.00(≥1) 66.97% 66.86% 2.02 0.49

(61.55 to 71.99) (64.58 to 69.06) (1.83 to 2.24) (0.42 to 0.58)(≥2) 49.54% 85.41% 3.40 0.59

(44.01 to 55.09) (83.65 to 87.02) (2.90 to 3.98) (0.53 to 0.66)(≥3) 36.09% 93.28% 5.37 0.69

(30.92 to 41.58) (91.98 to 94.39) (4.28 to 6.74) (0.63 to 0.74)(≥4) 23.24% 96.50% 6.63 0.80

(18.85 to 28.28) (95.49 to 97.29) (4.84 to 9.09) (0.75 to 0.84)(≥5) 17.74% 98.33% 10.65 0.84

(13.84 to 22.41) (97.58 to 98.86) (6.93 to 16.37) (0.80 to 0.88)(≥6) 12.23% 99.48% 23.66 0.88

(8.98 to 16.40) (98.98 to 99.75) (11.59 to 48.29) (0.85 to 0.92)(≥7) 9.17% 99.83% 53.24 0.91

(6.37 to 12.97) (99.45 to 99.96) n/a (0.88 to 0.94)(≥8) 6.12% 99.83% 35.49 0.94

(3.87 to 9.44) (99.45 to 99.96) n/a (0.91 to 0.97)(≥9) 3.36% 100.00% n/a 0.97

(1.78 to 6.11) (99.73 to 100) n/a (0.95 to 0.99)(≥10) 2.14% 100.00% n/a 0.98

(0.94 to 4.55) (100 to 100) n/a (0.96 to 0.99)(≥11) 1.22% 100.00% n/a 0.99

(0.39 to 3.32) (100 to 100) n/a (0.98 to 100)(≥12) 0.92% 100.00% n/a 0.99

(0.24 to 2.88) (100 to 100) n/a (0.98 to 1.00)(>12) 0.00% 100.00% n/a 1.00

N=2068

n/a, Not available.95% CIs are shown in parentheses.

Figure 4 Receiver operating characteristics (ROC) curves forManchester Children’s Early Warning System (ManChEWS) and PennineAcute Trust (PAT) Paediatric Observation Priority Score (PAT-POPS).

Cotterill S, et al. Emerg Med J 2016;33:756–762. doi:10.1136/emermed-2015-204647 761

Original article on A

ugust 31, 2020 by guest. Protected by copyright.

http://emj.bm

j.com/

Em

erg Med J: first published as 10.1136/em

ermed-2015-204647 on 11 A

pril 2016. Dow

nloaded from

score highly in the categories of behaviour (if they were agitatedor in pain) or nurse’s judgement (a child with a severelydeformed fractured limb).

A PAT-POPS score of 2 or more was associated with correctidentification of 50% of children who were admitted from theED, and a PAT-POPS score of less than 2 was associated withcorrect identification of 85% of children who were dischargedfrom the ED. While this is a useful tool to aid decision-makingon admissions in our population of patients, at a score of 2 ormore, about half of children who were admitted would havebeen discharged, and 15% of those who were discharged homewould have been admitted. Understanding which of thesepatients are more likely to be missed is a factor that should beaddressed in future research. The results in this study comparefavourably to previous studies. In a review of the performanceof 10 Paediatric Early Warning Scores (PEWS) in predicting hos-pital admission of 17 943 children aged 0–16 years attending anED, the area under the curve of the ROC curves (AUROCs) forthe 10 PEWS ranged from 0.56 (95% CI 0.055 to 0.58) to 0.68(0.66 to 0.69).18

The NHS Institute PEWS is a valid tool with good diagnosticaccuracy in recognising children at risk of serious and life-threatening deterioration at triage in the ED, but further workis needed to determine whether other subjective measures haveany value in paediatric early warning tools.19 In a study of chil-dren aged 0–16 years a PEWS score of ≥2 had a sensitivity of37% and a specificity of 88%. The authors concluded thatPEWS is of limited value in predicting admission (in a triagesetting) in a population of undifferentiated disease. However, alow PEWS score has a high specificity, that is, a patient scoring<2 is unlikely to need admission.20 PAT-POPS has a largerAUROC than any of the 10 PEWS tested in the study, and itcontains more subjective criteria than PEWS, making it an idealcandidate for future work to resolve the concerns that werehighlighted in the above two studies.

The mean PAT-POPS score was 0.9 on a scale of 0–18 whichmay indicate that some of the items used to calculate the scoremay not be especially relevant, particularly at the lower end ofthe scale. Future work should investigate the weighted contribu-tions of each of the components of PAT-POPS to the total scoreand whether there are some of the current components thatcould be either modified or removed without detrimental effecton the sensitivity and specificity reported in this study.

This initial study was undertaken as a service evaluation of anew tool and the intention is that, after this preliminary work, amulticentre study will be undertaken to investigate further. Werecommend further research in two areas. First, refinement andvalidation of PAT-POPS should take place to ensure the variouscomponents in the score are combined together to make themost effective tool. Second, multicentre validation would beuseful to determine if the effects found at our site are replicatedelsewhere. In carrying out this further work there is a clear needto identify a gold standard outcome measure setting out clearlythe criteria upon which the appropriateness, or otherwise, of anadmission have been determined.

KEY LEARNING POINTS FROM THIS STUDY▸ PAT-POPS has the potential to be an effective tool for use in

deciding admission of children from the ED.▸ Further research could refine the tool and enhance its pre-

dictive accuracy.▸ There is a need to identify a robust gold standard outcome

measure for hospital admission.

Twitter Follow Sarah Cotterill at @cotterillsarah1 and Andrew Rowland at@DrAndrewRowland

Acknowledgements The authors thank Christopher Hoare (Clinical Auditor) forsubstantial input into the data entry and initial data analysis as well as Dr DamianRoland, Consultant in Paediatric Emergency Medicine, for allowing them to modifyPOPS and for providing advice and guidance throughout this research project. Theauthors also thank four anonymous reviewers for their valuable comments.

Contributors SC wrote the first draft, undertook the statistical analysis, finalisedthe manuscript and revised the paper in response to reviewer comments. AGRconceived and led the research project, had substantial input into the writing, andrevised the paper in response to reviewer comments. JK co-led the data collection,reviewed the draft manuscript and had substantial input into database design anddata entry. HL co-led the data collection, reviewed the draft manuscript and hadsubstantial input into database design and data entry. MK facilitated the literaturesearch, contributed to data entry and reviewed the draft manuscript.

Funding Pennine Acute Hospitals NHS Trust Endowment funds covered some of thecosts of this study.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Open Access This is an Open Access article distributed in accordance with theCreative Commons Attribution Non Commercial (CC BY-NC 4.0) license, whichpermits others to distribute, remix, adapt, build upon this work non-commercially,and license their derivative works on different terms, provided the original work isproperly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

REFERENCES1 Pearson GA, ed. Why children die: a pilot study 2006; England (South West, North

East and West Midlands), Wales and Northern Ireland. London: CEMACH, 2008.2 Chapman SM, Grocott MP, Franck LS. Systematic review of paediatric alert criteria

for identifying hospitalised children at risk of critical deterioration. Intensive CareMed 2010;36:600–11.

3 Ferguson S, Stark M, Madar J. Paediatric early warning system (PEW)—a model forimproved recognition and management of the critically ill. Arch Dis Child 2003;88:A30–2.

4 Joshi V, Barber R, Yates R. ManChEWS: Royal Manchester Children’s Hospital earlywarning score. Crit Care 2011;15(Suppl 1):507.

5 Edwards ED, Powell CV, Mason BW, et al. Prospective cohort study to test thepredictability of the Cardiff and Vale paediatric early warning system. Arch Dis Child2009;94:602–6.

6 Akre M, Finkelstein M, Erickson M, et al. Sensitivity of the pediatric early warningscore to identify patient deterioration. Pediatrics 2010;125:e763–9.

7 Parshuram CS, Duncan HP, Joffe AR, et al. Multicentre validation of the bedsidepaediatric early warning system score: a severity of illness score to detect evolvingcritical illness in hospitalised children. Crit Care 2011;15:R184.

8 Parshuram CS, Hutchison J, Middaugh L. Development and initial validation of thebedside paediatric early warning system score. Crit Care 2009;13:R135.

9 Duncan H, Hutchison J, Parshuram CS. The Pediatric Early Warning System score:a severity of illness score to predict urgent medical need in hospitalized children.J Crit Care 2006;21:271–8. .

10 Roland D, Coats TJ. An early warning? Universal risk scoring in emergencymedicine. Emerg Med J 2011;28:263.

11 ManChEWS website https://sites.google.com/site/emmedapps/home/manchews(accessed 2 Feb 2016).

12 Roland D, Davies F, Lewis G. Determining the effect of objective and subjectivecriteria on a risk assessment tool in a Children’s Emergency Department. EmergMed J 2011;28:A12–13.

13 Leicester POPS website http://www2.le.ac.uk/departments/cardiovascular-sciences/research/cardiovascular-physiology-and-pathophysiology/emergency-medicine-group/research/pemla/pops/pops-front-sheet (accessed 2 Feb 2016).

14 Roland D, Davies F, Coats T. The Paediatric Observation Priority Score (POPS):outcomes of 24000 patients. Arch Dis Child 2014;99:A24.

15 Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ 2004;329:168.16 DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or

more correlated receiver operating characteristic curves: a nonparametric approach.Biometrics 1988;44:837–45.

17 Kelly J, Rowland AG, Cotterill S, et al. The Paediatric Observation Priority Score(PAT-POPS): a useful tool to predict likelihood of admission from the emergencydepartment. Emerg Med J 2013;10:877–8.

18 Seiger N, Maconcochie I, Oostenbrink R, et al. Validity of different pediatric earlywarning scores in the emergency department. Pediatrics 2013;132;4:e841–50.

19 Sinitsky LL, Reece A. Effort of breathing is not an important parameter in aPaediatric Early Warning Scoring System. Arch Dis Child 2013;98:A6.

20 Bradman K, Maconochie I. Can paediatric early warning scores be used as a triage tool inpaediatric accident and emergency. Eur J Emerg Med 2008; 15:359–60.

762 Cotterill S, et al. Emerg Med J 2016;33:756–762. doi:10.1136/emermed-2015-204647

Original article on A

ugust 31, 2020 by guest. Protected by copyright.

http://emj.bm

j.com/

Em

erg Med J: first published as 10.1136/em

ermed-2015-204647 on 11 A

pril 2016. Dow

nloaded from