Translating Research Into Practice:

Theory, Evidence, Planning

Ralph Gonzales, MD, MSPH

UCSF

April 1, 2008

Course Overview (1)

• Course Goals– Learn how to design & evaluate interventions– Learn some of the evidence base for specific types of

interventions– Develop and refine an intervention protocol

• Deconstruct Interventions-- Who Changes?– Population/Community/Public– Patients– Providers– Systems/Organizations

Sequential Approach to Intervention Design

A B

Diagnosis Treatment

Understanding the Problem

Designing the Solution

Explanatory Models

Intervention/

Behavior Change Models

Health Care Interventions

Understanding Behavior

Intervening on Behavior

Theory Evidence Planning

Course Overview (2)

• Homework– Required Reading before class– Protocol development

• Exercise assigned after each class• Due following Sunday night (earlier is better)

• Grades– Based on homework; participation in seminars; final

protocol and final presentation (equally-weighted)

TICR Professional Conduct StatementClarifications for this class

• I will maintain the highest standards of academic honesty

• I will neither give nor receive aid in examinations or assignments unless such cooperation is expressly permitted by the instructor

• I will conduct research in an unbiased manner, reports results truthfully, and credit ideas developed and work done by others

• I will not use answer keys from prior years• I will write answers in my own words, and, when

collaboration is permitted, acknowledge collaborators when answers are jointly formulated

Course Schedule

• April 1 – May 13: patient, physician and system; and program evaluation lectures (n=7)

• May 20: peer review and feedback of protocols (small groups using class time)

• May 27: analytical designs and power• June 3: final protocol presentations

– 15 minutes x 14 = 7 + 7… two separate groups…– Funding agencies

• CHCF (M Laws); RWJF (?Disparities for Change Agent?); AHRQ ( ); UCSF Medical Center (Adler)

Introductions

• Name

• Division/Department

• Health Outcome you want to (ultimately) improve

Translating Research Into Practice: The Birth of T2

T1 T2

“I think that we have to ask ourselves whether much of the output of biomedical science is getting

lost in translation?” –C.Lenfant, NEJM 2003;349:868-74. Former Director NHLBI.

NIH Roadmap Initiative-translating discoveries into health

NIH Roadmap Initiative-translating discoveries into health

Zerhouni E. Science 2003.

NIH Roadmap Initiative-translating discoveries into health

Westfall JM et al, JAMA 2007

T2 = Quality

Condition (n=25) Recommended Care, %Senile Cataract 79%Breast Cancer 76%Prenatal Care 73%….Dyspepsia/Ulcer Disease 33%Atrial Fibrillation 25%Hip Fracture 23%Alcohol Dependence 11%Overall Average 55%

Quality of Health Care• Donabedian A. JAMA 1988;260:1743-8

Structure Process Outcomes

Community Characteristics

Delivery System Characteristics

Provider Characteristics

Population Characteristics

Health Care Providers-Technical Processes-Interpersonal Processes

Public & Patients-Access-Acceptance-Adherence

Health Status

Functional Status

Satisfaction

Mortality

Cost

Provider Behavior is at the Core…

Health Care Provider

Technical Processes of Care

Interpersonal Processes of Care

Behavior-testing-diagnosis-treatment-procedures-referrals

Outcome(in future)

Knowledgejudgment

skill

empathysensitivity

Patients

System

Institute of Medicine Six Dimensions of Health Care Quality

Health Care That Is…• Safe• Timely• Effective• Efficient• Equitable• Patient-Centered

Institute of Medicine, Committee on Quality Health Care in America. Crossing the quality chasm: a new health system for the 21st century. 2001. Washington DC, National Academy Press.

Priority areas for national action: transforming health care quality. Adams K and Corrigan JM. 2003. Washington DC, National Academy Press.

1st Order Strategies to Improve Health Care Quality

-measurement

Practice Variation = Poor Quality

http://www.dartmouthatlas.org/

Overuse

Underuse

Misuse

1st Order Strategies to Improve Health Care Quality

-accountability

National Committee for Quality Assurancewww.ncqa.org

HEDIS Effectiveness of Care Measures 2003, comm

• Beta-blocker post MI 94%• Cancer screening

– Breast 75%– Cervical 82%– Colorectal 47%

• Chlamydia screening 30%• Cholesterol screening 79%• HbA1c testing 85%• Eye exams in diabetes 49%• Controlling hypertension (<140/90) 62%• LDL < 100 after 60 days of MI 48%

CMS/JCAHOHospital Quality Measures

• Management of AMI– Aspirin on arrival and discharge– Beta-blockers on arrival and discharge– Lysis within 30 min of arrival– PCI within 90 min of arrival– ACE or ARB for LVSD– Management of CHF– ACE-inhibitor at discharge

• Management of CHF– ACE or ARB for LVSD– LVSD evaluation (echo)– Discharge counseling– Tobacco cessation

• Management of Pneumonia• Surgical Care Improvement

– Antibiotics within 1 hour of surgery; appropriate abx; d/c after 24 hours– DVT prophylaxis

Hospital Comparewww.hospitalcompare.hhs.gov

 

                                                                                                              

AVERAGE FOR ALL REPORTING HOSPITALS IN THE UNITED STATES

AVERAGE FOR ALL REPORTING HOSPITALS IN THE STATE OF CALIFORNIA - NORTHERN & CENTRAL

KAISER FOUNDATION HOSPITAL

MARIN GENERAL HOSPITAL

NOVATO COMMUNITY HOSPITAL

 

                                                                                                              

   

 Top Hospitals represents the top 10% of hospitals nationwide. Top hospitals achieved a 97% rate or better.

  75% 

  69% 

  37% 

  82% 

  85% 

Percent of Surgery Patients Who Received Treatment To Prevent Blood Clots Within 24 Hours Before or After Selected Surgeries to Prevent Blood Clots

Link Quality Gap to “Outcome Gap”

What is the Patient’s Role?

T1

You can invest large NIH $$ to develop new drugs… but they are no good if patients don’t have access [structure], physicians prescribe [process], and patients accept and

comply with treatment [process]

T2

Cross-Sectional Model of Health Care Behavior

-adapted from Kleinman et al, and Donabedian

Clinician Factors -knowledge -attitudes -behavior -heuristics -specialty

Patient Factors -knowledge -attitudes -behaviors -SES -health literacy

System Factors -local culture -access to care -regulatory policies -organizational culture

Likelihood of Physician Behavior (Process)

Translating to Whom?

Green L et al. NEJM 2001;344:2021-25

Getting Started-Just Do It?

Impact of CMS/JCAHO on Quality Improvement Activities

• Why was CMS/JCAHO measures so much more effective at stimulating QI action than NCQA/HEDIS?– CMS linked to reimbursement/accreditation– vs. HEDIS was voluntary

• little/no evidence that employers paid much attention to HEDIS in determining health plans

– Hospital vs. health plan control over providers and QI investment

• Stimulated lots of measurement and investment, but no methods/roadmap

Book Ends

“Just as in the rest of medicine, we must pursue the solutions to quality and safety problems in a way that does not blind us to harms, squander scarce resources, or delude us about the effectiveness of our efforts.” –Auerbach A et al, NEJM 2007;357:608-13

“The effectiveness of these systems is sensitive to an array of influences: leadership, changing environments, details of implementation, organizational history, and much more. In such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect.” –Berwick D. JAMA 2008;299:1182-84

2nd Order Strategies to Improve Health Care Quality

Continuous Quality Improvementand PDSA Cycles

Don Berwick; Institute for Healthcare Improvement

A Roadmap for Designing Theory-Based Interventions

• Needs Assessment

• Understanding the Problem within a Theoretical Framework

• Designing Multifaceted Intervention within a Theoretical Framework

• Process and Outcome Evaluation of Intervention’s Impact

• Refine and Repeat

PRECEDE-PROCEDE

Donabedian = QualityRogers = Adoption/Uptake

Which innovations in health care get spread and adopted into practice?

1. Better than status quo.2. Compatible with current values and needs of potential

adopters.3. Gain exposure to potential adopters, ideally from a

trusted and respected source.4. Time for “S-curve” to materialize.

i.e., early adopters early-late majority laggards5. Consistent social norms and opinion leaders.

Rogers EM. Diffusion of Innovations, 4th ed. New York (The Free Press), 1995.Bodenheimer T. The Science of Spread. California Health Care Foundation. 2008.

Needs Assessment

Generic Approach to Needs Assessment

• What behavior to measure/assess?– Is it important? Strong evidence for association with

outcomes?– Use judgment and experts… FINER

• Frequency?– Are there trends over time?

• Distribution?– Does it vary by patient, physician or system factors?– Use theory or conceptual framework…

• Quality? – Case-mix and risk adjustment?– Do you conclude underuse, overuse or misuse?

IOM Priority Areashttp://www.iom.edu/?id=19752

Asthma

Care coordination

Children with special health care needs

Diabetes

End of life

Cancer screening

Frailty associated with old age

Hypertension

Immunization

Ischemic heart disease

Major depression

Medication management

Nosocomial infections

Obesity (emerging area)

Pain control in advanced cancer

Pregnancy and childbirth

Self-management/health literacy (cross-cutting area)

Severe/persistent mental illness

Stroke

Tobacco-dependence treatment in adults

Ralph’s Research Questions

c. 1995

“We need to decrease overuse of antibiotics for acute respiratory infections”

c. 1997

“How often are antibiotics prescribed for adults with uncomplicated acute bronchitis?”

Penicillin-Resistant S. pneumoniae, 1979-2000 (US)

05

1015202530

1979 1982 1985 1988 1991 1994 1997 2000

% N

onsu

scep

tible

Intermediate Resistance High level resistance

1979-1994: CDC Sentinel Surveillance Network1995-2000: CDC Active Bacterial Core Surveillance (ABCS) System Emerging Infections Program

European Surveillance of Antimicrobial Utilization-Goosens et al. Lancet 2005;365:579-87.

Correlation Between Antibiotic Utilization and Antibiotic Resistance in Community Bacteria

Acute Respiratory Illnesses Account for 75% of Total Ambulatory Antibiotics

cold/URI16%Sinusitis

12%

Other24%

Otitis Media21%

Pharyngitis12%

Bronchitis15%

McCaig, 1995

Measuring the Quality Gap

Condition Visits Rx Rate Bact. Prev Abx Excess

Otitis Media 13 x 106 76% 65% 1.1 million

Sinusitis 11 x 106 70% 40% 3.5 million

Pharyngitis14 x 106 62% 25% 5.2 million

Bronchitis 13 x 106 59% 10% 6.5 million

URI/cold 25 x 106 30% 5% 6.2 million

TOTAL 76 x 106 54% 25% 22.5 million

Gonzales R, et al. JAMA, 1997;278:901-904

Gonzales R, et al. Clinical Infectious Diseases, 2001, 33:757-62

www.epibiostat.ucsf.edu/courses/RoadmapK12/PublicDataSetResources/

National Data Sources-Public and Readily Available

Data Sources–Patient BRFSS; NHIS; NHANES; MEPS

–Provider NAMCS; NHAMCS

–Delivery system NCQA; Hospital Compare

–Community Dartmouth Atlas

Administrative Claims DataAdministrative data collected as a result of “claims”

submitted by physicians/practices for reimbursement.

• Medicare– No pharmacy data

• Medicaid– Enrollment rollercoaster

• Integrated Delivery Systems– Generalizability

• Hospital Networks (Premier)• Managed Care Organizations

Computerized health records are becoming a new resource for quality and outcome measurement… but can you think of some of the limitations??

UCSF Resources for Assessing Quality

SFGHTHREADS;

Community Health Center Network

-A. Bindman

VAMCEHR-Vista-M. Chren

UCSF Moffitt-Long Hospitals

UB-92 databaseUCARE?

-A. Auerbach

SFGHTHREADS;

Community Health Center Network

Kaiser PermanenteAdministrative

Database; EHR -J. Selby

Summary

• The need for theory- and evidence-based strategies to improve translation of new treatments/tests into improved health outcomes is needed.

• Improving health outcomes depend on changing patient, provider, system and community behaviors.

Summary

• Theory-based intervention design starts with a comprehensive needs assessment– Current behavior patterns, quality gaps, and

association b/n quality gap and outcome gap

• Become familiar with secondary data sources for your particular area

“We should avoid claiming

for our capacity to assess quality

either too little or too much…

Steer the middle course”

-Donabedian, 1983

HOMEWORK: To be turned-in by Sunday night (earlier is better!)

1. Identifying the target behavior that your intervention seeks to improve:--Who is being assessed and intervened upon?--What is the optimal performance pattern for this behavior?--How strong is the evidence linking this behavior to improved health?--Identify some of the factors that influence this behavior using the patient-clinician-system matrix.

2. Identify an existing source of data that one can use to estimate the current performance level of your target behavior, and conduct a PubMed search to provide citations of previous work on quality or performance measurement of this behavior.

Performance Measures

Clinician & System• Treatment• Testing• Referrals• Counseling• Communication

Patient• Adherence• Testing• Self Care• Service

Utilization• Lifestyle

Behaviors