INDEPTH Network Effectiveness and Safety Studies Platform (INESS)

Introduction to Systems Effectiveness Modules

Don de Savigny

INDEPTH Scientific Advisory Committee

Swiss Tropical Institute, Basel

Pune, 28 October, 2009

INESS systems effectiveness objective

To assess the effectiveness, and determinants of effectiveness, of new malaria treatments in real world health systems.

Challenge

For INDEPTH DSS Sites…

To move beyond population health observatories to include a health system observatory function

To link population health and health behaviours to health services and to health system behaviours

= 37%

98%

Driving with the brakes on:How interventions lose traction in health systems

Example of ACT anti-malarial treatment in Rufiji DSS in 2006

Efficacy

X Access

X Diagnostics

X Provider compliance

X Patient adherence

Effectiveness

X 95%

X 95%

X 70%

X 60%Health system factors

Averages mask inequities

X 40%

X 90%

X 75%

X 60%

= 16%Poorest quintile

Data source: IMPACT Tanzania. Effectiveness data are actual. Poorest quintile estimates are hypothetical

What does this mean?

Presently more traction can be gained by removing health system bottlenecks than by improving the efficacy of new drugs.

INESS Technical approach for systems effectiveness Seven linked study modules provide the ingredients

for the effectiveness estimation:

Module Task team facilitator

1 Access STI

2 Diagnostic targeting CDC

3 Provider compliance CDC

4 Patient Adherence STI

5 Community acceptability CDC

6 Contexts and other effects STI

7 Costs and cost effectiveness SPH

Level

HH

HF

HF

HH

Community & HF

District & HH

District, HF & HH

INESS: Understanding barriers to effectiveness

Therapeutic efficacy

Targeting Actual PracticeComplianceAccess Adherence Effectiveness

Costs

HH HHHF HF

Module 1. Access

Main purpose: determine proportion of cases needing to seek care

that actually gain physical access to a point of provision

Quick overview:

Household surveys of fevers in prior two weeks Determines who was able to access authorized provider within 24h Determines reasons for choices and failed access Analyzes across time, space, socio-economic quintiles and provider

characteristics

Module 1. AccessMore details: DSS Total Population Monitoring via three core questions for every DSS household

Any fever in prior two weeks If yes, who (name, permanent ID) Did he/she take an antimalarial

Provides annual pattern of fever burden

DSS Household Access Sample Survey for in-depth assessment of care seeking and access on sample of those with fever (on PDAs) Sample size ~ 21,000 per year 2 hh per routine DSS enumerator per week requiring full interview Modified Malaria Indicator Survey instrument to identify:

ACT provider Delay and sequence of care seeking Whether any diagnostic test done for the ACT Whether and what treatment(s) obtained Whether full ACT course continuing or completed Costs of episode

RDT conducted and referral if needed Sample size ~ 1,690 per year

Powered to provide estimate +/-5% of proportion of RDT +ve febrile individuals having access to a source of ACT within 24 and 48h in both rainy and dry seasons by equity quintile.

Module 2 & 3. Diagnostic targeting & Provider compliance

Main purpose: determine the proportion of cases having access that are

correctly diagnosed / classified determine the proportion of correctly diagnosed cases

that are provided with the correct treatment

Quick overview:

Health facility / provider surveys Sampled at peak and low seasons Exit interviews with gold standard diagnostic Determines the drug and instructions provided or prescribed Assesses stock-outs and quality of drugs on hand Identifies cohorts for adherence follow-up survey at home

Module 2 & 3. Diagnostic targeting & Provider compliance

More details: Stratified sampling of ACT providers Sample size: 1,750 patients per year over two seasons All patients presenting as initial illness to sampled ACT provider on

day of survey Gold standard diagnostic Patient exit interview Pharmacy and supply inventory Health worker interview

Module 4. Patient adherence

Main purpose: Estimate proportion of patients who receive treatment who

use it as intended; and the proportion who are satisfied with the treatment

Quick overview:

Household survey Sample of exit subjects from Module 3 followed at home on day after last

scheduled dose (plus filter paper blood sample) Standard interviews for adherence and acceptability Further follow-up and filter paper blood at day 28 (and 42 depending on

ACT) Gold standard diagnostic available

Sample size: AdherenceThree levels of adherence: High (complete):65% of users with treatment failure of 5% Medium: 25% of users with treatment failure of 30% Low: 10% of users with treatment failure of 50%

Calculations Sample size required to detect a difference in treatment failure rate between the

two smallest groups, medium and low. With following parameters Confidence level: 95% Power 80% Ratio unexposed (medium adherence)/ Exposed (low adherence)

= 25% / 10% = 2.1 Prevalence of disease (treatment failure rate) in Exposed group: 50% Rate ratio = 50% / 30% = 1.67 We need 175 in medium adherence group and 70 in low adherence group. As the low group is expected to be 10% of all, we will need a total of 700 patients

to be followed through to the last day. This would have to be corrected upwards to account for the losses. Perhaps to 1000 patients per treatment for each drug.

Module 4. Patient adherence

More details: Sample size: 400 per season Visited one day after expected end of treatment course Asked about:

Doses taken on each day, individually Time specificity limited to morning, noon, afternoon, night How drugs were taken (with food, drink, etc) Vomiting and specifics

Pills remaining and packaging examined Filter paper blood sample taken 28 day interview and filter paper blood sampling visit scheduled

Module 5. Community acceptance

Main purpose: Examine the social, cultural and behavioural factors

that facilitate or impede uptake and adherence to new ACTs when introduced

Quick overview:

Community survey of three different populations Persons having a recent malaria fever episode (45-50 interviews) Adult men & women living in DSS area (15 FGDs per year) ACT providers (15-20 interviews)

Two communities <5km and two communities >5km from ACT

Module 6. Contexts and additional effects

Main purpose: Estimate the contribution to reduced morbidity &

mortality.

HMIS document reviews for trends and patterns in: Proportion of fevers recorded as malaria (OPD, IPD) Severe anemia Incidence of severe malaria Proportion requiring transfusion

DSS database and VA review for trends in: All cause and malaria specific mortality Health seeking prior to malaria death from verbal autopsy ITNs and IRS coverage

District plan and budget reviews for trends in: Health system changes Malaria expenditures as a share of all expenditure

Other contextual data (rainfall, EIR, molecular markers for resistance) Repeat therapeutic efficacy (100 patients)

Module 7. Overall effectiveness and costs

Main purpose: Determine the effectiveness, and the

determinants of effectiveness

Putting it all together Determine overall population effectiveness by equity quintile Determine the efficacy losses, and where the greatest losses occur Determine the costs of change, comparative financial costs, and

expected cost-effectiveness

Systems effectiveness: 20 IndicatorsElement of effectiveness Indicator

Access •Proportion of people with fever who have sought contact with a provider who should have the drug •Proportion of people with fever who seek care from other providers

Availability •Proportion of providers with the product in stock•Proportion of time product is in stock

Targeting accuracy •Proportion of malaria positive patients correctly diagnosed/classified by health providers

Compliance (health system and worker)

•Proportion of prescriptions which are correct (in accordance with manufacturer’s or MOH guidelines)

Adherence (individual) •Proportion of people who receive product and take as prescribed

Acceptability •Proportion of people who are satisfied with the tested antimalarial (qualitative assessment) •Proportion of people actually opting for the tested antimalarial

Other measures of effectiveness (including sensitivity of drug overtime)

•Parasitological cure rate (clearance of the initial parasite infection by day 7, persisting at D28, with PCR correction and/or in vitro and molecular markers as proxies for these measures optional for sites with capacity to measure them) •Parasite and anaemia prevalence•Blood drug level •Proportion of cases recorded as malaria in health facilities (outpatient + admissions) •Incidence of severe malaria and malaria-related anemia •Proportion of malaria cases requiring blood transfusions •Mortality rate (all causes, malaria-specific)

Related malariologic parameters that could influence findings

•Entomological Inoculation Rate (EIR)•Coverage of other malaria-control interventions

Costs / cost-effectiveness (based on a standardized cost tracking system)

•Incremental financial costs of drugs policy (costs of drugs + costs of other activities required to change policy)•Costs per clinical outcome

Systems Effectiveness Task Teams

Will assist with: Development of field protocols Piloting protocols in initial sites / countries Developing training and capacity strengthening approaches General oversight on module performance Data management & analysis

Thank you

District expenditure shares – all strategies

District absolute annual per capita expenditure – communicable diseases

Rufiji District 2007

Estimating District ACT requirements