Facility-Specific Employee Influenza Vaccination Data for 2008-09: Options and Considerations for Public Postings

Kate Cummings, MPHHealthcare Associated Infections Program

California Department of Public Health

Goals of the Ideal Mandatory Reporting & Disclosure Program1

• Maximize accuracy of data collection• Standardize methodology for data collection

and analyses• Minimize costs to hospitals and governmental

agencies• Produce data that are valid, fair to hospitals,

and useful to consumers

1 Edmond MF In: Hospital Infections, 5th ed., 2007.

CDC Healthcare Infection Control Practices Advisory Committee (HICPAC)1

• “Mandatory public reporting…is intended to enable stakeholders, including consumers, to make more informed choices on healthcare issues”

• Public reports must– Convey scientific meaning in a manner that is useful and

interpretable to diverse audiences– Must clearly communicate limitations of data and methods– Data should be assessed for validity/precision prior to release

1 Guidance on public reporting of health-care associated infections: Recommendations of the Healthcare Infection Control Practices Advisory Committee Am J Infect Control 2005;33:217-26

Selected Assumptions Underlying Public Reporting1

• Publicly reported healthcare quality data are valid• Consumers will use publicly reported data• Consumers are willing and able to change their site of

care• Consumers who use publicly reported data will make

decisions that will improve their care• Market forces derived from public reporting will provide

incentive for hospitals to improve quality• Positive outcomes will outweigh negative unintended

consequences

1 Edmond MB, Bearman GML, Mandatory Reporting in the US, an Example to Follow? J Hosp Infect 2007Jun;65 (2):182-8.

Opportunities

• Vaccination of health care personnel has been proven effective1

• The measure is unambiguous1; does not require risk adjustment

• Public reporting may increase vaccination rates (no published information on effectiveness)

1 Guidance on public reporting of health-care associated infections: Recommendations of the Healthcare Infection Control Practices Advisory Committee Am J Infect Control 2005;33:217-26

Potential constraints

“Developers of reporting systems should avail themselves to established and proven methods of collecting and reporting surveillance data…… publicly reported HAI rates can mislead stakeholders if inaccurate information is disseminated….”

1Guidance on public reporting of health-care associated infections:

Recommendations of the Healthcare Infection Control Practices Advisory Committee Am J Infect Control 2005;33:217-26

What is Surveillance?

The ongoing systematic collection and analysis of data and the provision of that information for use in disease prevention and control.

Established surveillance methods

• Clear goal (s)• Clear, uniform case definition (s)• Case ascertainment (case-finding) with consistent

inclusion and exclusion criteria• Consistent surveillance catchments (population under

study)• Standardized data collection and reporting instruments• Data quality assurance• Analytic strategies that address limitations to ensure

appropriate interpretation • Disseminate results

Interpreting Facility-Specific Comparisons

• Once posted, consumers must consider whether facility-specific percentages differ:– Distortion (systematic error)– Chance (precision)– Truly different

• Present percentages in a way that discloses potential limitations and allows appropriate interpretation

Option 1: Publicly Post a Report of State Level Data, Only

• Does not draw attention to facility-specific rates but overall picture of the state– Data are older - may not

represent the facility real time; limitations may affect validity

• Less likely to have unintended consequences associated with inaccurate information

• No access to facility-specific rates to inform healthcare decisions– Are consumers likely to use

reported data to make healthcare decisions?

– Are consumers able to choose site of care?

• No public/market forces to provide incentive for hospitals to report accurate, timely data or improve vaccination rates

Pros Cons

Option 2: Publicly Post Facility-Specific Data Listed in Alphabetical Order

• Access to facility-specific rates to inform healthcare decisions

• May create public/market forces to provide incentive for hospitals to report accurate, timely data or improve vaccination rates

• Because of data limitations, positive outcomes may not outweigh negative unintended consequences– Data limitations become

amplified at the facility-specific level

– Example: misclassification can falsely decrease or increase the rate

– Inaccurate data may mislead consumers

Pros Cons

Option 3: Publicly Post Facility-Specific Data Listed in Rank Order By Percentage Vaccinated

• Rank is clear including outliers• Target goals or benchmarks are

noted– California average – The Healthy People 2010: 60%– The proposed HP 2020: 90%

• Facility rates to inform healthcare decisions

• May create public/market forces to provide incentive for hospitals to report accurate, timely data or improve vaccination rates

• Attention to outlier facilities– May be misclassified as an outlier

in either tale due to data limitations previously described

• Because of data limitations, positive outcomes may not outweigh negative unintended consequences

Pros Cons

Option 4: Publicly Post No Data

• Ensures no unintended consequences associated with reporting inaccurate information

• Consumers and other stakeholders will have no information

• No public/market forces to provide incentive for hospitals to report accurate, timely data or improve vaccination rates

Pros Cons

• The 2008-09 data were not collected using established surveillance methodology and may not be valid– Data limitations impact facility rates; this affects our options for

public reporting– Opportunities outweighed by potentially harmful unintended

consequences? • misleading consumers, misdirecting quality improvement efforts,

loss of credibility and public trust, diversion from true mission

• Strongly consider releasing only aggregate data with full disclosure of limitations without specifying facility specific rates

• Future data must be collected using standardized methodology

• Given limitations, consider using 2008-09 as a test year; use lessons learned to plan for future data

Summary