PROMS Monthly Oct 2010

1 PROMs: provisional analysis of the experimental data, April 2009 to October 2010

Patient Reported Outcome Measures (PROMS), Pre and Post Operative Data,

April 2009 to October 2010, West Midlands

Introduction

Patient Reported Outcome Measures (PROMs) measure quality from the patient perspective. Initially

the PROMs programmes cover four clinical procedures: groin hernia surgery, hip replacement, knee

replacement and varicose vein surgery. PROMs calculate the health gain (as reported by the patient)

after surgical treatment using pre and post operative surveys.

This report updates PROMs data to October 2010 for all West Midlands’ providers. At this time the data

remains experimental nationally, therefore this report does not draw any conclusions on the variation in

the quality of services. The aim of the report is to keep clinicians and commissioners informed about the

programme and to seek their views on the process and how WMQI can assist in the application of

PROMs in improving the quality of services.

Previous PROMs reports by the WMQI can be accessed from:

http://www.wmqi.westmidlands.nhs.uk/clinical-measures/proms/wmqis-work-on-proms/

Data Collection

Before a patient undergoes one of the four PROMs procedures, the provider should offer the patient a

PROMs questionnaire (i.e. pre-operative questionnaire) for completion. The guidance states that this

should happen in the interval between the patient being passed fit for surgery and the treatment taking

place. Completion of the questionnaire is voluntary for the patient.

The completed pre-operative PROMs questionnaires are then transferred to the DH contractor

responsible for collating the information, where the questionnaires are linked securely to the Hospital

Episode Statistics (HES)1 and National Joint Registry (NJR)2 databases.

After three or six months, depending on procedure, the DH contractor posts out the follow-up post-

operative questionnaire to the patient’s home. Once the form has been completed by the patient and

returned, it is electronically scanned and linked with the pre-operative data along with HES and NJR, if

applicable.

Definition of PROMs Measures

Two types of PROMs measures are reported: (i) the EQ-5D system for general health and (ii) the

condition-specific measures, Oxford hip and knee score and Aberdeen varicose vein score.

1 See http://www.hesonline.nhs.uk/Ease/servlet/ContentServer?siteID=1937 for details of HES. 2 See http://www.njrcentre.org.uk/njrcentre/default.aspx for details of NJR.

http://www.wmqi.westmidlands.nhs.uk/clinical-measures/proms/wmqis-work-on-proms/

http://www.hesonline.nhs.uk/Ease/servlet/ContentServer?siteID=1937

http://www.njrcentre.org.uk/njrcentre/default.aspx


EQ-5D Index Score

The EQ-5D descriptive system comprises the following five dimensions describing the patient’s general

health:

mobility

self-care

usual activities

pain/discomfort

anxiety/depression

Each dimension has three levels: no problems, some problems, and severe problems. EQ-5D health

states are then converted into a single summary index score up to 1(best health) by applying a formula

that essentially attaches weights to each of the levels in each dimension.

Oxford Hip/Knee Score

The Oxford Hip/Knee Score comprise of 12 questions relating to the patient’s experience of pain, ease of

joint movement and ease of undertaking normal domestic activities such as walking or climbing stairs.

All questions are presented denoting least (or no) symptoms scoring 4 and those representing greatest

severity scoring 0. The individual scores are then added together to provide a single score between 0

(worst) and 48 (best).

Aberdeen Varicose Vein Score

The Aberdeen Varicose Veins Questionnaire (AVVQ) is scored from 0 (best quality of life) to 100 (worst

quality of life).

Data Analysis

Case-mix adjusted average health gain in terms of the measures described above is estimated for each

provider. The adjusted measure takes into account the fact that organisations deal with patients with a

differing case-mix, e.g. age, sex, co-morbidity. Further information regarding the methodology can be

found at http://www.northgateproms.co.uk/documents.html

When interpreting the organisational-level results, it is important that the patterns shown are taken to

be a starting point for further investigation rather than giving a definitive conclusion on organisational

performance. Extreme results may not be down to clinical reasons. They could also be caused by

random variation (irregular and erratic fluctuations or chance factors that, in practical terms, cannot be

anticipated, detected, identified, or eliminated); by data quality issues; or by differences.

For further information and a national summary see

http://www.hesonline.nhs.uk/Ease/servlet/ContentServer?siteID=1937&categoryID=1295

http://www.northgateproms.co.uk/documents.html

http://www.hesonline.nhs.uk/Ease/servlet/ContentServer?siteID=1937&categoryID=1295


Groin Hernia

Data Completeness

Table 1 shows the statistics on the number of submitted PROMs questionnaires for groin hernia

procedures (both pre and post operative) by provider for the period between April 2009 and October

2010. Please see the ‘Data Collection’ section on Page 1 of the report for details of the ‘eligible HES

episodes’, ‘pre-operative questionnaires (Pre-op Qs)’, ‘linked questionnaires (Qs linked to HES episodes)’

and ‘post-operative questionnaires (Post-op Qs)’.

Table 1 PROMs data completeness (April 2009 – October 2010) – groin hernia

Providers3

Eligible HES

episodes4

Pre-Op Qs

Qs linked to HES

episodes5

Post-Op Qs

returned to date

6

% Post-Op Qs

returned out of

episodes7

England 108045 57556 40642 34679 32.1%

Burton Hospitals 223 117 85 77 34.5%

Walsall Hospitals 457 302 218 168 36.8%

Sandwell & West Birmingham Hospitals 764 287 238 153 20.0%

University Hospital of North Staffordshire 734 187 140 136 18.5% The Royal Wolverhampton Hospitals 567 354 273 226 39.9%

Worcestershire Acute Hospitals 1117 460 363 297 26.6%

University Hospital Birmingham 686 293 219 173 25.2%

South Warwickshire General Hospitals 422 305 271 203 48.1%

University Hospitals Coventry & Warwickshire 853 391 299 260 30.5%

Mid Staffordshire General Hospitals 382 137 100 84 22.0%

Dudley Group of Hospitals 731 345 258 220 30.1%

Shrewsbury & Telford Hospital 1149 867 663 561 48.8%

Heart of England NHS 1194 846 573 426 35.7%

BMI - The Droitwich Spa Hospital 54 69 38 50 92.6%

South Staffordshire PCT 182 101 97 76 41.8%

Hereford Hospitals 324 292 227 158 48.8%

The Midlands NHS Treatment Centre 275 257 162 132 48.0%

Spire South Bank Hospital 64 142 46 98 153.1%

The George Eliot Hospital 357 232 180 144 40.3% Data source: Hospital Episode Statistics (HES) / PROMs, The NHS Information Centre for Health and Social Care

3 Only providers with case-mix adjusted scores calculated are shown in this table. For Other providers, please see Table 5 Appendix. 4 The data from April 2009 to March 2010 has been finalised and will not change. Data from April 2010 onwards is currently provisional. 5 These figures are based on questionnaires linked to HES episodes using an algorithm that compares a number of different combinations of patient identifiers. This methodology may be revised or enhanced for future publications. The counts are based upon the pre-operative questionnaire completion date. 6 The number of records affects the statistical reliability of the PROMs measures and caution should be exercised when looking at data for trust with less than 25 records submitted. 7 The counts are based upon the pre-operative questionnaire completion date.


EQ-5D Index Score

Figure 1 shows the case mix adjusted change in the average EQ-5D index scores reported (between pre

and post-operation) for each West Midlands’ providers of groin hernia operations. Description of the

EQ-5D index score can be found from the Definition of PROMs Measures section on Page 1 of this

report.

Figure 1

Data source: Hospital Episode Statistics (HES) / PROMs, The NHS Information Centre for Health and Social Care

Figure 1 shows that:

Case-mix adjusted changes in EQ-5D score for all 19 West Midlands’ providers were positive (i.e.

improvement in health). For fifteen of the providers, the improvement was statistically

significant (i.e. confidence intervals do not cross 0);

There was no statistically significant difference between any of the West Midlands’ providers

and the national average on this measure.

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

Case mix adjusted changes (pre and post operation) in average EQ-5D Index Score with confidence intervals, Groin Hernia Operations, PROMs, Apr 2009 - Oct2010

worse

better


Hip replacement

Data Completeness

Table 2 shows the statistics on the number of submitted PROMs questionnaires for hip replacement

procedures (both pre and post operative) by provider during the period between April 2009 and October




Table 2 PROMs data completeness (April 2009 – October 2010) – hip replacement*

Providers3

Eligible HES

episodes4

Pre-Op Qs

Qs linked to HES

episodes5

Post-Op Qs

returned to date

6

% Post-Op Qs

returned out of

episodes7

England 103297 76694 56766 42656 41.3%






Robert Jones & Agnes Hunt Orthopaedic & District Hospital 1702 1602 1288 934 54.9%

Royal Orthopaedic Hospital 1531 1572 1169 804 52.5%




The George Eliot Hospital 388 370 312 209 53.9%

The Royal Wolverhampton Hospitals 508 370 335 197 38.8%

University Hospital of North Staffordshire 672 770 509 397 59.1%



Worcestershire Acute Hospitals 1010 644 519 350 34.7% Data source: Hospital Episode Statistics (HES) / PROMs, The NHS Information Centre for Health and Social Care

*Please see Page 3 for all the footnotes in this table


EQ-5D Index Score

Figure 2 shows the case mix adjusted change in the average EQ-5D index scores reported (from pre-

operation to post-operation) for all West Midlands’ providers. Description of the EQ-5D index score can

be found from the Definition of PROMs Measures section on Page 1 of this report.

Figure 2



Case-mix adjusted changes in the average EQ-5D score for all 16 West Midlands’ providers were

positive (i.e. improvement in health) and the improvement was statistically significant (i.e.

confidence intervals do not cross 0);



0.200

0.250

0.300

0.350

0.400

0.450

0.500

0.550

Ch

ange

in E

Q-5

D

Changes (pre and post operation) in ADJUSTED EQ-5D Index Scores with confidence intervals, Hip Replacement Operations, PROMs,

Apr 2009 - Oct2010

worse

better


Oxford Hip Score

Figure 3 shows the adjusted change in the average Oxford Hip Score reported (from pre-operation to

post-operation) for each West Midlands’ providers. Description of the Oxford Hip score can be found

from the section Definition of PROMs Measures on Page 1 of this report.

Figure 3



Case-mix adjusted changes in the average Oxford hip score for all 16 West Midlands’ providers

were positive (i.e. improvement in health) and the improvement was statistically significant (i.e.


None of the West Midlands’ providers was significantly different from the national average on

this measure.

12.0

14.0

16.0

18.0

20.0

22.0

24.0

26.0

Ch

ange

in O

xfo

rd H

ip S

core

Changes (pre and post operation) in average Oxford Hip Score with confidence intervals, Hip Replacement Operations, PROMs,

Apr 2009 - Oct2010

worse

better


Knee replacement

Data Completeness

Table 3 shows the statistics on the number of submitted PROMs questionnaires for knee replacement





Table 3 PROMs data completeness (April 2009 – October 2010) – knee replacement*

Providers3

Eligible HES

episodes

4

Pre-Op Qs

Qs linked to HES

episodes5

Post-Op Qs

returned to date

6

% Post-Op Qs

returned out of

episodes7

England 116728 91391 62348 48853 41.9%






Robert Jones & Agnes Hunt Orthopaedic & District Hospital 1709 1648 1241 920 53.8%

Royal Orthopaedic Hospital 1425 1279 1038 632 44.4%




The George Eliot Hospital 550 528 432 283 51.5%


University Hospital of North Staffordshire 640 694 411 390 60.9%






EQ-5D Index Score


operation to post-operation) for each West Midlands’ providers of knee replacement operations.

Description of the EQ-5D index score can be found from the Definition of PROMs Measures section on

Page 1 of this report.

Figure 4



Case-mix adjusted changes in the average EQ-5D score for all 16 West Midlands’ providers were

positive (i.e. improvement in health) and the improvement was statistically significant (i.e.


University Hospital of North Staffordshire are significantly worse than the England average

0.100

0.150

0.200

0.250

0.300

0.350

0.400

0.450

0.500

Ch

ange

in E

Q-5

D

Changes (pre and post operation) in ADJUSTED EQ-5D Index Scores with confidence intervals, Knee Replacement Operations, PROMs,

Apr 2009 - Oct2010

worse

better


Oxford Knee Score

Figure 5 shows the case-mix adjusted change in the average Oxford Knee Score reported (from pre-

operation to post-operation) for each West Midlands’ providers. Description of the Oxford Knee score

can be found from the section Definition of PROMs Measures on Page 1 of this report

Figure 5



Case-mix adjusted changes in the average Oxford knee score for all 16 West Midlands’ providers

were positive (i.e. improvement in health) and the improvement was statistically significant (i.e.


University Hospital of North Staffordshire is significantly worse than the England average and

the West Midlands average; Sandwell & West Birmingham are significantly worse than the

England average.

Burton Hospitals are significantly better than the England average and the West Midlands

average.

8.0

10.0

12.0

14.0

16.0

18.0

20.0

Ch

age

in O

xfo

rd k

nee

sco

re

Changes (pre and post operation) in Oxford Knee Score with confidence intervals, Knee Replacement Operations, PROMs, Apr 2009 - Oct2010

worse

better


Varicose Vein

Data Completeness

Table 4 shows the statistics on the number of submitted PROMs questionnaires for varicose vein





Table 4 PROMs data completeness (April 2009 – October 2010) – varicose vein*

Providers3

Eligible HES

episodes4

Pre-Op Qs

Qs linked to HES

episodes5

Post-Op Qs

returned to date

6

% Post-Op Qs

returned out of

episodes7

England 54266 23420 18913 12381 22.8%






The Midlands Nhs Treatment Centre 80 50 44 33 n/a


University Hospital Birmingham Nhs Foundation Trust 902 200 172 102 11.3%






EQ-5D Index Score


operation to post-operation) for each West Midlands’ providers of varicose vein operations. Description

of the EQ-5D index score can be found from the Definition of PROMs Measures section on Page 1 of this

report.

Figure 6



Case-mix adjusted changes in EQ-5D score for all of the West Midlands’ providers were positive.

The improvement was statistically significant for nine of the ten providers (i.e. confidence

intervals do not cross 0);



-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

Ch

ange

in E

Q-5

D

Changes (pre and post operation) in ADJUSTED EQ-5D Index Scores with confidence intervals, Varicose Vein Operations, PROMs, Apr 2009 - Oct2010

worse

better


Aberdeen Varicose Vein Score

Figure 7 shows the case mix adjusted change in the average Aberdeen varicose vein scores reported

(from pre-operation to post-operation) for each West Midlands’ providers. Description of the Aberdeen

varicose vein score can be found from the Definition of PROMs Measures section on Page 1 of this

report.

Figure 7



Case-mix adjusted changes in the average Aberdeen Varicose Vein Score for all 11 of the West

Midlands’ providers were negative (i.e. improvement in health). The improvement was

statistically significant for seven of the providers (i.e. confidence intervals do not cross 0);



-25.000

-20.000

-15.000

-10.000

-5.000

0.000

5.000

10.000

15.000

Ch

ange

in A

be

rdee

n S

core

Changes (pre and post operation) in Aberdeen Varicose Vein Score with confidence intervals, Varicose Vein Operations, PROMs, Apr 2009 - Oct2010

better

worse


Next steps

WMQI will be publishing the West Midlands data on a monthly basis on our website

(http://www.wmqi.westmidlands.nhs.uk/clinical-measures/proms/), along with relevant commentary

on the methodologies and the PROM scores.

Please send any comments you have to [email protected] or use the feedback form on our

website (http://www.wmqi.westmidlands.nhs.uk/about-wmqi/feedback/).

Produced by

WMQI

NHS West Midlands

St Chads Court

213 Hagley Road

Edgbaston

Birmingham

B16 9RG

tel: 0121 695 2460

email: [email protected]

web: www.wmqi.westmidlands.nhs.uk

twitter: @wmqi

http://www.wmqi.westmidlands.nhs.uk/clinical-measures/proms/

mailto:[email protected]

http://www.wmqi.westmidlands.nhs.uk/about-wmqi/feedback/

http://www.wmqi.westmidlands.nhs.uk/


Appendix Data quality for providers which are excluded from the case-mix adjusted

analysis Table 5 PROMs data completeness (April 2009 – October 2010) – groin hernia, other providers*

Providers Eligible

HES episodes

4

Pre-Op Qs

Qs linked to HES

episodes4

Post-Op Qs

returned to date

5

% Post-Op Qs

returned out of

episodes6

England 108045 57556 40642 34679 32.1%

West Midlands Hospital 122 39 30 24 19.7%

Rowley Hall Hospital 124 40 30 24 19.4%

BMI - The Meriden Hospital 62 27 19 18 29.0%

Worcestershire PCT 60 36 29 22 36.7%

Spire Little Aston Hospital 58 75 45 48 82.8%

Spire Parkway Hospital 53 74 9 56 105.7%

Shropshire County PCT 43 15 13 12 27.9%

BMI The Edgbaston Hospital 28 30 13 16 57.1%

Nuffield Health, Wolverhampton Hospital 21 88 16 47 n/a

Birmingham Children's Hospital 20 * * * n/a

Nuffield Health, Hereford Hospital 19 79 10 40 210.5%

Nuffield Health, Shrewsbury Hospital 13 50 8 25 192.3%

Table 6 PROMs data completeness (April 2009 – October 2010) – hip replacement, other providers*

Providers Eligible

HES episodes

4

Pre-Op Qs

Qs linked to HES

episodes5

Post-Op Qs

returned to date

6

% Post-Op Qs

returned out of

episodes7

England 103297 76694 56766 42656 41.3%


Rowley Hall Hospital 59 36 31 25 n/a

Kidderminster Nhs Treatment Centre 45 0 * n/a n/a





Bmi - The Droitwich Spa Hospital 29 62 23 26 89.7%

Bmi The Edgbaston Hospital 24 33 13 12 50.0%

Bmi - The Meriden Hospital 22 19 14 * n/a

Nuffield Health, Wolverhampton Hospital 10 93 12 50 500.0%

Nuffield Health, Hereford Hospital 8 90 6 62 775.0%

University Hospital Birmingham Nhs Foundation Trust 6 0 0 0 0.0%


Table 7 PROMs data completeness (April 2009 – October 2010) – knee replacement, other providers*

Providers Eligible

HES episodes

4

Pre-Op Qs

Qs linked to HES

episodes5

Post-Op Qs

returned to date

6

% Post-Op Qs

returned out of

episodes7

England

University Hospital Birmingham Nhs Foundation Trust * 0 0 0 n/a

Nuffield Health, Hereford Hospital * 71 * 45 n/a

Nuffield Health, Shrewsbury Hospital * 46 * 26 n/a

Kidderminster Nhs Treatment Centre 116 0 27 n/a n/a




Rowley Hall Hospital 45 33 28 13 28.9%


Bmi - The Droitwich Spa Hospital 35 77 22 35 100.0%

Bmi - The Meriden Hospital 27 16 11 * n/a

Bmi The Edgbaston Hospital 12 35 6 * n/a

Nuffield Health, Wolverhampton Hospital 6 142 * 40 666.7%

Table 8 PROMs data completeness (April 2009 – October 2010) – varicose vein, other providers*

Providers Eligible

HES episodes

4

Pre-Op Qs

Qs linked to HES

episodes5

Post-Op Qs

returned to date

6

% Post-Op Qs

returned out of

episodes7

England 54266 23420 18913 12381 22.8%

South Staffordshire Pct * * * * n/a

Rowley Hall Hospital * 0 0 0 n/a

University Hospital Of North Staffordshire Nhs Trust 290 51 35 32 11.0%

South Warwickshire Nhs Foundation Trust 98 7 6 * n/a

Mid Staffordshire Nhs Foundation Trust 95 14 12 9 9.5%

George Eliot Hospital Nhs Trust 59 33 26 23 39.0%

Burton Hospitals Nhs Foundation Trust 31 19 10 9 29.0%

Shropshire County Pct 17 8 8 7 41.2%




Documents

PROMS Monthly Oct 2010