PROCESS vs. WA State SCS Study A Comparison of Study Design, Patient Population, and Outcomes August 29,2007

PROCESS vs. WA State SCS Study

A Comparison of Study Design, Patient Population, and Outcomes August 29,2007

2 | MDT Confidential

Study Design

• A non-randomized, prospective cohort study to describe costs, complications and patient outcomes among WA state Workers’ Compensation CBLP claimants in three treatment groups (SCS group, Pain Clinic group, Usual Care group)

• Observational• Followed up to 24 months

• A prospective, randomized (1:1), controlled, multi-center trial to test the efficacy and cost-effectiveness of spinal cord stimulation (SCS) plus conventional medical management (CMM) vs. CMM alone for patients with failed back surgery syndrome (FBSS).

• Crossovers permitted at 6 months.

• Followed up to 24 months

WA State PROCESS


Inclusion Criteria

• 18-60 years of age*• Current WA state WC fund claim for

a back injury (any duration)*• 1 to 3 previous open lumbar spine

surgeries (during current claim)*• Current receipt of time loss

compensation• Pain radiating into one or both legs

for more than 6 months• Radicular pain greater than axial pain• Average leg pain in the last month

rated as 6 or greater on a 0-10 scale

• 18+ years of age

• Neuropathic pain predominantly in legs (>50%)

• VAS Pain intensity > 50 mm

• Symptoms >6 months following

– At least one anatomically successful surgery for herniated disc

WA State PROCESS

* Criteria revised due to slow enrollment. Original criteria included age 18-55, claim <3 years’ duration, and 1 or 2 previous surgeries


Exclusion Criteria

• Previous SCS surgery• Diagnosis of diabetes or cancer• Unable to speak English or Spanish• Progressive motor deficit or bony

deformity or any contraindication for surgery

• Other clinically significant or disabling chronic pain condition

• Expected inability to receive or operate the SCS system

• History of:

– Coagulation disorder

– Lupus

– Diabetic neuropathy

– Rheumatoid arthritis

– Ankylosing spondylitis

• Active psychiatric disorder

• Other condition known to affect perception of pain

• Inability to evaluate treatment outcome

• Life expectancy <1 year

• Existing or planned pregnancy

WA State PROCESS


WA State Cohorts: What We Know About Cohort Assignment

• Injured workers who are authorized by DLI for a SCS procedure are compared to two comparison groups

• Injured workers in this group were told that SCS would be covered if they participated in the study (was not otherwise)

• Workers potentially eligible for SCS were initially identified at clinic visits by health care providers who performed the SCS procedure. They were screened and send to DLI for administrative review and then U of W research team contacted worker for remainder of screening/invite

• Injured workers who meet the same clinical criteria as used for the SCS group and who have not been referred for authorization for SCS, but who have been referred to a multi-disciplinary pain clinic

• Workers were identified from DLI administrative databases at the time they were approved for the pain clinic evaluation

• They were screened and send to DLI for administrative review and then U of W research team contacted worker for remainder of screening/invite

SCS cohort Pain Clinic cohort Usual Care cohort• Injured workers who

meet the same clinical criteria as used for the SCS group, but who have not been referred for DLI authorization for SCS (Usual Care group)

• Individuals were randomly selected from DLI administrative files

• U of W research team contacted worker for all screening/invite

Note: Patients were already referred, which leaves the possibility of selection bias.


Primary and Secondary Endpoints

Primary:

Leg pain (NPRS), physical disability (RDQ), and use of opioid analgesics

Secondary:

Back pain intensity, assessment of leg pain vs. 1 year prior, current work status, types of therapy used, mental health (SF-36), complications, device use, cost (to be reported in future publications)

Primary: Proportion of patients achieving at

least 50% leg pain relief at 6 months (VAS)

Secondary: Improvement in back and leg

pain, HRQoL (EQ-5D, SF-36), functional capacity (ODI), change in use of pain medication and non-drug therapy, patient satisfaction with treatment, incidence of adverse events, cost (to be reported in future publications)

WA State PROCESS


Statistical Analysis

• No power calculations due to descriptive nature of study (post hoc noted)

• Target of 50 patients per group because the authors thought this number of patients could be recruited in a 1-year period

• Used both intent-to-treat and as treated principles

• ANOVA, chi-square and Kruskal-Wallis used at baseline

• ANCOVA with adjustment for select covariates used at follow-up

• Power calculation performed to determine adequate sample size

• Used both ITT and as treated principles

• Chi-square and t-tests used at baseline

• Linear and logistic regression with select covariates used at follow-up– Also used step-wise, LOCF

method, interaction terms, and completed sensitivity analysis

WA State PROCESS


Patient Flow

SCS (n=61), Pain Clinic (n=217), Usual Care (n=295)


159 Enrolled



52 in SCS cohort

51 in Pain Clinic cohort


56 in Usual Carecohort

Unable to contact

Approached

Eligible

Refused

214 Screened

135 Eligible

79 Ineligible

35 Refused Participation

100 Randomized

52 Assigned to SCS 48 Assigned to CMM

WA State PROCESS

• PROCESS enrollments occurred between April 2003 and June 2005

• WA state enrollments occurred between December 2004 and June 2006

Ineligible


Demographics & Baseline Characteristics (1 of 3)

Baseline Characteristic WA State PROCESS

SCS (n=52) Pain Clinic (n=51)

Usual (n=56)

SCS

(n=52)

CMM

(n=48)

Age, mean (SD) 44.6 (7.9) 44.3 (6.0) 44.8 (9.4) 48.9 (10) 52.0 (10.7)

Gender, (% female) 21% 27% 20% 42% 56%

Employed* 2% 2% 9% 23% 21%

Compensation / Legal action 42% 43% 21% 10% 17%

Years since last surgery (SD) NA NA NA 4.7 (5.1) 4.6 (4.3)

Months since injury (median) 57 50 47 NA NA

Prior surgeries, mean (SD) 1.9 (1.0) 1.7 (0.8) 1.6 (0.9) NA NA

>1 prior surgery, n (%) NA NA NA 22(46) 28(54)

*PROCESS defined as FT or PT status; WA State defined as working or other (student/retired)





Usual (n=56)

SCS

(n=52)

CMM

(n=48)

Leg pain Intensity, mean (SD)^ 7.6 (1.0) 7.2 (1.1) 7.4 (1.1) 73.4 (14.0) 76.0 (13.0)

Back pain intensity, mean (SD) 6.0 (1.9) 6.1 (1.6) 6.4 (2.0) 44.8 (23.2) 54.5 (24.3)

RDQ, mean (SD) 20.9 (2.2) 20.3 (2.4) 20.1 (2.3) NA NA

Oswestry, mean (SD) NA NA NA 57.4 (12.5) 55.2 (15.4)

SF-36 Bodily Pain mean (SD) NA NA NA 15.3 (11.8) 17.4 (12.9)

SF-36 Phys funct mean (SD) NA NA NA 24.7 (16.4) 21.9 (15.1)

SF-36 Mental hlth mean (SD) 34.4 (12.6) 33.5 (11.6) 35.1 (10.5) NA NA

^ Leg and back pain measured at baseline visit for PROCESS and in past month for WA state. 0-100 scale for PROCESS and 0-10 scale for WA state.





Usual (n=56)

SCS

(n=52)

CMM

(n=48)

Opioid use 85% 80% 77% 79% 81%

NSAID use 15% 26% 21% 87% 94%

Antidepressants NA NA NA 60% 74%

Anticonvulsants 27% 14% 32% 63% 74%

Physical therapy NA NA NA 79% 87%

Injections/blocks NA NA NA 60% 70%

Education/counseling NA NA NA 23% 34%

Bilateral leg pain (%) 48% 38% 31% 37% 33%


Results: Primary Outcome at 6 Months

PROCESS: 6 Months (ITT analysis)

CMM Group (n=44) SCS group

(n=50)

Between group difference

p-value

>50% leg pain relief - n (%) 4 (9%) 24 (48%) P<0.001*

* Indicates a statistically significant difference

^ Adjusted logistic regression model


Results: Primary Outcome at 12 Months

PROCESS: 12 Months (modified ITT analysis –as pts were assigned)

CMM Group (n=41) SCS group

(n=47)

Between group difference p-value

>50% leg pain relief - % 3 (7%) 16 (34%) P=0.005*

WA State: 12 Months (ITT analysis)^

SCS

(n=40)

Pain Clinic (n=47)

Usual

(n=51)


p-value

>50% leg pain relief - n (%) 3 (8%) 1 (2%) 7 (14%) 0.09

>2 point increase in RDQ – n(%) 13 (33%) 21 (45%) 20 (39%) 0.39

< daily opioid use – n(%) 9 (23%) 10 (21%) 17 (33%) 0.52

All three criteria – n(%) 0 (0%) 0 (0%) 2 (4%) Not calculated

* Indicates a statistically significant difference

^ Adjusted logistic regression model


Percentage of Patients Receiving Their Assigned Treatment: 6-months

WA State PROCESS

• 24/52 patients assigned to SCS received the therapy; 20 were trialed and not implanted due to failed trial; 8 had not been trialed

• 31/51 patients assigned to Pain Clinic group received the therapy several times per week; 20/51 patients received the therapy < several times per week

• 54/56 patients assigned to Usual Care group completed the follow-up; 2 received SCS; 2 unable to contact.

• 48/52 patients randomized to SCS received the therapy; 4 were not implanted due to a failed trial

• 48/48 patients randomized to CMM received the therapy


Results: Secondary Outcomes

WA State: 12 months^

SCS

(n=40)

Pain Clinic

(n=47)

Usual

(n=51)


p-value

Back pain intensity, mean (SD) 6.9 (1.9) 6.8 (1.9) 6.4 (2.0) 0.50

SF-36 Mental hlth, mean (SD) 34.3 (14.2) 33.5 (12.3) 35.2 (12.1) 0.99

Leg pain vs. 1 yr ago• much/somewhat better• about the same• much/somewhat worse

23%

35%

42%

19%

36%

45%

33%

39%

27%

0.34

ADLs vs. 1 yr ago• much/somewhat better• about the same• much/somewhat worse

15%

53%

32%

15%

38%

47%

20%

47%

33%

0.56

Working* 8% 4% 10% 0.59

^ Adjusted analysis

* Versus 2%, 2% and 9% at baseline for 6%, 2% and 1% RTW


Results: Secondary Outcomes

PROCESS: 6 months (ITT analysis)^

CMM

(n=44)

SCS

(n=50)

Between Group Difference p-value

Leg pain VAS mean (SD) 66.6 (24.0) 39.9 (26.3) <0.00001*

Back pain VAS mean (SD) 51.6 (26.7) 40.6 (24.9) 0.008*

SF-36 bodily pain mean (SD) 19.5 (12.9) 33.0 (20.9) 0.0001*

SF-36 physical function mean (SD) 21.8 (16.2) 38.1 (23.0) 0.0005*

Oswestry DI mean (SD) 56.1 (17.9) 44.9 (18.8) 0.0002*

Opioid use n (%) 31 (70%) 28 (56%) 0.20

Return to work n (%) 1/33 (3%) 4/36 (11%) 0.36

Patient satisfaction with tx n (%) 22 (50%) 43 (86%) 0.0003*

^ 12-month data not yet finalized, 6-month data for secondary outcomes will be reported in forthcoming manuscript

* Achieved statistical significance


Brief Critical Review (1 of 2)*Category WA State PROCESS

Study Design Assessment

• The research question is useful, but the design is not appropriate to draw conclusions about the effectiveness of the therapy• Use of composite endpoints appear nowhere else in the SCS literature and <daily opioid use as an outcome is unconventional.• Definitions for primary pain and disability outcomes reasonable

• The research question is useful and the study design is appropriate (experimental) to draw conclusions about efficacy.• No composite endpoints used.• Definitions for primary pain outcome reasonable

Internal Validity Assessment

• Bias could possible explain study results as study design was not experimental and variables known to influence outcome were not all controlled

• Experimental design helps eliminate potential of bias explaining study results.

Selection Bias • Groups were appropriate, concurrent and similar, but no randomization or concealment of group allocation• Data on how patient’s qualified for referral to groups not presented

• Groups were appropriate, concurrent and similar. Patients randomized, but study was not blinded.• Concealment of randomization procedure described

Performance Bias • Double blinding not employed• “Intervention” and comparator groups reasonable• Don’t know what co-interventions were used prior• Low rate of crossover occurred (~11% mostly from SCS to Pain Clinic and Usual Care)•Low compliance to assigned treatment; large number of pts assigned to SCS never trialed/implanted

• Double blinding not employed• Intervention and comparator groups reasonable• Know co-interventions used at baseline• High rate of crossover (10% SCS to CMM and 73% CMM to SCS), but not until after 6 months

* Based on Delfini checklist: http://www.delfini.org/Delfini_Tool_StudyValidity_Short.doc


Brief Critical Review (2 of 2)

Category WA State PROCESS

Attrition Bias • Minimal missing data points / good follow-up rate (~93% at 12 months)

• Minimal missing data points / good follow-up rate (88% at 12 months)

Assessment Bias • Not clear whether assessors were blinded for telephone interviews; disinterested party• No statistical significance, but post-hoc power analysis indicated sufficient power• ITT analysis conducted

• Use of assessors not detailed; patients completed diaries• Statistically significant differences with a priori power calculations completed• ITT analysis conducted

Usefulness Assessment • Clinically significant area for study, but no sufficient benefit size seen for any one of the three therapies vs. another

• Clinically significant area for study as well as sufficient benefit size seen for SCS vs. CMM (statistically and clinically meaningful changes)

External Validity • Results valid for other Workers’ Compensation populations, not for CBLP patients as a whole.

• Results valid for non-WC CBLP patients in Europe and Canada; similar demographics and baseline characteristics to US patients suggest validity in US as well.

Patient Perspective • Benefits of SCS for WC patients vs. Pain Clinic and Usual Care not apparent. Cost data not yet presented. Patient satisfaction not measured.

• Benefits of SCS for non-WC patients vs. CMM apparent. Cost data not yet presented. Patient satisfaction with SCS high.

Provider Perspective • Relevant to providers treating WC patients. Provider satisfaction not measured.

• Relevant to providers treated non-WC patients. Provider satisfaction not measured.

* Based on Delfini checklist: http://www.delfini.org/Delfini_Tool_StudyValidity_Short.doc


Level of Evidence: Based on Oxford CEBM Tool*

• 2b (at best) defined as an individual cohort study.

• If the cohort study is considered to be of poor quality, the level would be 4.

• Poor quality is defined as a study that failed to clearly define comparison groups and/or failed to measure exposures and outcomes in the same (preferably blinded), objective way in both exposed and non-exposed individuals and/or failed to identify or appropriately control known confounders and/or failed to carry out a sufficiently long and complete follow-up of patients.

• 1b defined as an individual RCT with a narrow confidence interval

• A wide confidence interval example provided by CEBM is when an ARR in an RCT is not statistically significant but whose confidence intervals fail to exclude clinically important benefit or harm. This does not apply to PROCESS.

• If PROCESS had <80% follow-up, it would be considered low quality and have a grade of 2b. This does not apply. PROCESS had 93% of patients contributing data at the 6-month primary endpoint and 88% at 12 months.

WA State PROCESS

* http://www.cebm.net/index.aspx?o=1025


• PROCESS is a RCT generating level 1b evidence in support of SCS for CBLP (FBSS) over CMM alone. Statistically significant, clinically meaningful improvements demonstrated for pain, function and QOL.

• The WA State SCS study is a prospective, comparative cohort study generating level 2b evidence (at best) and potentially level 4 evidence of no significant benefit of SCS for CBLP (FBSS) over Pain Clinic or Usual Care.

• PROCESS has external validity for non-WC FBSS patients, while the WA State study has external validity for WC FBSS patients only.

Conclusions (1 of 2)


• In the WA State study, the absence of information about how patients qualified for referral to SCS or Pain Clinic in addition to the non-experimental design opens the door for bias.

• In the WA State study, attempting to improve pain, function and return to work in a cohort of WC patients injured on average 4 years prior to study enrollment could be considered a near fatal flaw.– We know from the literature that WC patients have poor outcomes

and are unlikely to RTW– WC patients have the ability to be closely tracked/managed in

terms of health services utilization. They are, in many ways, a captive population that could have been studied in a far more effective (i.e. experimental) manner.

– The observational/naturalistic approach to the WA State study leaves the reader unable to draw conclusions about the effectiveness of one therapy option over another.

Conclusions (2 of 2)

Documents

PROCESS vs. WA State SCS Study A Comparison of Study Design, Patient Population, and Outcomes August 29,2007