Embed Size (px)
Citation preview
AHSC AFP Accountability: Information and Feedback Session
May 4, 2007Jenkins Lecture Theatre
Sunnybrook Health Sciences CentreToronto
2
Introductions
Dr. Chris Morgan, Chair, AHSC AFP Accountability Expert PanelPanel Members:
Nan Brooks, Chair, AHSC AFP Accountability Working GroupRami Rahal, Data Consultant, ASHC AFP Accountability Working Group
3
Meeting Format
Those attending via web cast will only see the speaker at the podium
Web cast attendees have been provided with an email address for submission of questions
The presenters will complete the presentation and take questions at the end
Participants in the audience are asked to raise their hand to indicate their intent to ask a question. Those participating by web cast are asked to e-mail their questions to [email protected] their questions will be placed in queue for response.
Questions from the audience will be repeated for the web cast viewers
4
Presentation Overview:
Presentation Objectives
Accountability Expert Panel Accountability overviewAccountability frameworkImplementation plan
Accountability Working GroupMandateLogic modelAccountability measures electionRecommendations
5
Presentation Objectives:• To provide an update on AHSC AFP accountability process
• Consultation to focus on principles and content more than implementation issues
• To obtain your input:
• Do the recommendations and measures have face validity?
• Are there additional measures you feel should be included?
• What is required to move forward to the next phase of the accountability framework?
• Will these recommendations fit with accountability requirements for other parts of the health system?
6
Background – AHSC AFP Initiative
• Problems with recruitment and retention in academic medicine• $75M annually awarded for academic stabilization through the 2000
Physician Services Agreement and distributed as Phase I AFP funding• 13 AFP agreements covering nearly 3,000 MD’s in Ontario AHSC’s• 2004 Physician Services Agreement - additional $150M ‘new
investment’ made available annually to enhance the AHSC AFP initiative and address clinical repair
• Appendix G of the agreement required establishment of an AHSC AFP Task Force to advise on:
• the development of a common AHSC AFP template agreement • a methodology for allocation and distribution of the new investment.
7
Accountability Expert Panel‘to advise on the measurement, accountability and reporting of deliverables and the methodology to be used to determine complement’
appropriate methodologies for measuring AHSC deliverablesstructures & processes for reporting measurementsframework on which to base accountability for clinical deliverables and delivery and outcome of services to the population
8
Accountability Expert Panel
The AEP reports to the Physician Services CommitteeMembers of the AEP:
Dr. Chris Morgan - ChairDr. Peter Munt - Queen’s/SEAMOMr. Mark Rochon - TRI/JPPC/PSC/OHAMr. Stewart Saxe - OMA counselDr. Andreas Laupacis- ICES /SMH (past member)
9
Common elements of accountability
answer for actions, decisions & outcomesclearly defined expectationsmeasurement…..but also evaluation and reportingbeing accountable implies having authority and adequate resourcesaudit function but also catalyst for changeconsequences – with emphasis on incentive
10
AHSC AFP Accountability Overview
Accountability mechanisms establish an agreed upon set of goals, deliverables and accountability measures
Accountability mechanisms provide a catalyst for positive change and improvement
The AEP recognized that the complexities of an AHSC require a multi-level view of accountability
11
AHSC AFPPaths of Accountability & Responsibility
Public/Ministry:Value for Money,Quality and Accessto the Community
AHSC AFPGovernance:Hospitals,Universities
Physicians
Patient/ClientPhysicians: Clinical,Teaching,Research,&Admin.
Delegation of Responsibility
Departments, Clinical Units :Manage TeamsProvide AgreedService
Reporting and Accountability
12
AHSC AFP Accountability Phases
Phase A: AHSC level accountabilityPhase B: Add departmental/specialty level accountabilityPhase C: Add physician level accountability if and when appropriate
The implementation of an accountability framework will be accomplished in phases to allow for the development of appropriate measures and reporting mechanisms:
13
Specific objectives for first phase
1. maintain clinical volumes at the AHSC’s2. maintain access (& be concordant with
wait time strategy etc.)3. build physician trust and buy in4. begin to learn how to go beyond OHIP
billing in measuring physician performance
14
Principles
1. Until such time as a new accountability framework has been developed, implemented and adequately evaluated, individual physician level clinical encounters will continue to be measured by OHIP reporting.
15
Principles2. The accountability framework will respect
the strategic priorities of the MOHLTC. It will also respect the existing accountabilities of:
the hospitals for clinical care, the universities for teaching and the academic departments and their associated practice plans for individual physician compensation
16
Principles
3. There will be common processes for review, reporting and disseminating results of performance measurements under the accountability framework. The results shall be transparent within and between the participating AHSC’s.
17
Principles
4. The accountability framework will acknowledge that the accountable party can be held responsible only for the deliverables over which it has authority, control and for which it has adequate resources.
18
Recommendations
1. The domains of: clinical services, education/scholarly activity, research and medical administration should all be addressed in the accountability agreement.
2. Performance measures, in particular those relating to clinical performance, will be tailored to individual specialties.
19
Recommendations
3. In developing the performance measures, and implementing a new accountability framework, there will be no attempt to assign a relative value to the various domains.
20
Recommendations
4. Both the AHSCs and the Ministry must commit fully to the development, evaluation and ongoing infrastructure (human, information technology, funding) required of a new accountability framework to quantify activities under the AFP.
There will be a commitment to iterative development of the performance measures under the accountability framework.
21
Recommendations
5. The accountability framework shall include a mechanism and process for dispute resolution.
silent re: ‘consequences’ at presenthow to make it a two way street?
22
Processes
to develop a new accountability framework that will, at the individual physician level, replace the existing accountability scheme embodied in the fee for service billing system
1. A provincial working group will be establishedwith representation from academic physicians, OMA, Ministry, academic hospitals and universities
23
Processes
2. Performance measures will be developed with the direct participation of the affected physician groups and under a common set of guidelines, mandate and specific consultative processes to be established by the AEP / steering committee.
specialty leaders / hospitals for clinical indicatorsuniversities for educational indicators
24
Accountability Working Group -TOR(1)
develop candidate measures in each of the 4 domains respecting the suggested selection criteriaperform the data analyses necessary for evaluating available AHSC performance measure optionsadvise on measures of service volume in a blended modelrecommend appropriate reporting mechanisms and approaches to ensure appropriate data quality and validity
25
Accountability Working Group-TOR(2)
provide advice to the AEP on mechanisms to ensure full consultation and input from all AHSC governance organizationsadvise the AEP on the best timing for the phased introduction of a comprehensive accountability frameworkadvise the AEP on communications related to development and implementation of accountability measures
26
Next Steps for AEP
AWG Report and feedback from today’s session will be considered by the AEPAEP will report recommendations to the Task Force and Physician Services CommitteeFinal report will be drafted and submitted to the Physician Services Committee
27
AWG MembershipMs. Nan Brooks (Chair), UHNMs. Alison Blair, Health Results Team for InformationDr. Jeff Bloom, Toronto WesternDr. Eric Cohen, Sunnybrook Mr. Jim Flett, London Health SciencesDr. Jasmin Kantarevic, OMAMr. John Lott, Kingston General/SEAMO Dr. Jane MacNaughton, Provider Services/OHIP Dr. Chris Morgan, Sunnybrook (ex-officio)Dr. Jeff Turnbull, Ottawa HospitalMr. Kevin Sulewski, McMaster UniversityMs. Jennifer Zelmer, CIHIAHSC AFP Project: Bernita Drenth, Brenda Edwards, Sandy Nuttall,
Susanne BjernoConsultant Support: Dr. Antoni Basinski, Rami Rahal
28
AWG Process Overview
Design accountability logic modelDesign accountability measure screening processShort list accountability measures in three categories:
PerformanceExplanatoryDevelopmental
Recommendations to the AEP for future refinement
29
Logic Model
Goals Resource Strategy Activity Output Outcome
WORK INTENDED RESULTS
30
Logic Model – Clinical Domain
Timely Access to Services
by Population
Clinical FTEs by Specialty
Clinical Centres of Excellence
Goals Resources Strategies Activities Outputs Outcomes
WORK INTENDED RESULTS
Volume, LOS of
Selected Diagnoses
Unplanned Readmit to
Hospital
Acute Patient Days
31
Goals Resources Strategies Activities Outputs Outcomes
WORK INTENDED RESULTS
Clinical Domain
Timely Access to Services by Population
Maintain/Improve Safety, Quality
and Outcomes of Care
Lead in Best Practices, Innovation
Clinical FTEs by Specialty
Bed Days Staffed
Number of Full Time Clinicians
Nursing, Allied
Health FTEs
OHIP Billings
Weighted Cases
Clinical Centres of Excellence
Unique Regional
RoleRehabilitation
Days
Ambulatory Care Volumes
Volume, LOS of Selected Diagnoses
Unplanned Readmission to Hospital
Acute Case Mix Index
Relative Wait Times for Selected Services
Attract/Retain the Best
Academic Physicians
Coordinate Hospital,
University, Physician
Resources to Meet
Deliverables
CCC Weighted
Patient Days
Mental Health Days
Emergency Room Visits
Average CTAS Level
Complex Continuing Care Case Mix Index
Surgical Cases
Acute Patient Days
ICU Days
32
AHSC AFP Accountability Logic Model
A logic model leads to a comprehensive and aligned set of accountability measures in each of the four AHSC activity domains:
ClinicalEducationResearchLeadership and Innovation
33
AHSC AFP Accountability Logic Model
The logic model ensured that measures were sought for all components of the model including:
Inputs and ResourcesActivities and OutputsOutcomes and Results Achieved
34
Accountability Measure Screening
The AWG began with more than 50 possible measures from various sources including existing AFPs
Implemented a screening process to ensure alignment with AHSC AFP goals and “usefulness”criteria
Conducted a feasibility assessment to ensure measures were based on high quality, timely data and established methodologies
35
Measure Selection Criteria: Support for Key AHSC Goals
1. Ensure strong/accountable AHSC governance
2. Coordinate hospital, university and physician resources to meet defined deliverables
3. Integrate physician practice and hospital/LHIN planning
4. Support system transformation and sustainability
5. Ensure timely access to key services by population
6. Lead in best practices, innovation and education
7. Better integrate research, clinical care, teaching and link to compensation
8. Maintain/improve safety, quality and outcomes of care
9. Attract/ retain the best academic physicians
36
Measure Selection Criteria: Useful
1. Does it follow best practice
2. Does it measure what it should
3. Is the data available/timely
4. Is the data of a high quality
5. Is the indicator within control
6. Is the indicator known to all (familiar)
7. Can the indicator be benchmarked over time and/or with peers
37
Potential Measure List
(various sources)
Measures
Selected
AHSC AFP
Accountability
Measure
Selection Process
Measure
Domains
Support AHSC Goals
Research
Useful
Leadership/InnovationClinical Teaching
38
Accountability Measure Categories
Performance Measures
Explanatory Measures
Developmental Measures
39
Accountability Measure Categories
Performance MeasuresIncluded in AHSC AFP measures templateAHSC Governance accountabilitySupport for template agreement negotiations
40
Accountability Measure Categories
Explanatory MeasuresA “tool kit” for interpretation of performance measures
Resources that must be in place to deliver on performance or support analysis of performance measure
Included in AHSC AFP measures template
41
Accountability Measure Categories
Developmental MeasuresIdentified by AWG but not yet feasible
Require data collection or methodology development
42
AHSC AFP Accountability Measures
Recommended Performance Measures:Total OHIP billings
Patient service volumes (weighted cases, weighted days, rehabilitation days, mental health days, emergency room visits, ambulatory care visits)
Number of medical trainees
Canadian Institute of Health Research (CIHR) funding
43
AHSC AFP Accountability MeasuresRecommended Explanatory Measures:
OHIP billings by main specialty groupVolumes (surgical cases, acute inpatient days, special care unitdays)Bed days staffed and in operationClinical support department full time equivalent staff (nursing and diagnostic)Relative wait times for hospital care for select servicesAcute case mix indexComplex continuing care case mix indexAverage Canadian Triage and Acuity Scale (CTAS) levelRelative hospital readmission rates
44
Developmental Measures Trainee satisfactionResearch FTEsPeer reviewed grantsRegistered clinical trials, inventions, patentsDevelopment/implementation of evidence based best practicesPhysician participation in planning, policy and patient care improvement processes
45
Developmental Measures (cont’d)Clinical FTEsNumber of full time physiciansAHSC accreditation statusAccredited CME program deliveryLicensing and exam pass ratesPhysician education of non-physiciansMedical Trainee Days
46
Accountability Measures Template(Sample for Education and Research Domains)
EDUCATION
2004/05 Actual
2005/06 Actual
2006/07 Projected
2007/08 Budget
2007/08 Target Minimum
2007/08 Target Maximum
Comment 2004/05 Actual
2005/06 Actual
2006/07 Projected
2007/08 Planned
Accountability Indicator Link
A4. Count of Medical Trainees by AHSC8a Undergraduate (U3/4)8b Residents8c Fellows
RESEARCH
2004/05 Actual
2005/06 Actual
2006/07 Projected
2007/08 Budget
2007/08 Target Minimum
2007/08 Target Maximum
Comment 2004/05 Actual
2005/06 Actual
2006/07 Projected
2007/08 Planned
Accountability Indicator Link
A6. Research Funding9 CIHR Funding
Domain:Explanatory MeasuresAccountability Measures
Accountability Measures Explanatory MeasuresDomain:
47
AHSC AFP AWG RECOMMENDATIONS
48
1. Accountability Measures Template
Include the Accountability Measures Template in the 2007 AHSC AFP template agreement
The template will serve as a guideline for measuring the impact of available funding on furthering the stated AHSC goals
The on-going development of the accountability structure should be based on incentives for good performance, not penalties
49
2. Implement an Accountability Cycle
AHSC AFP evaluates results, implement change or stay the course
Establish MOHLTC/AHSC AFP
expectations and calculate AHSC
performance indicatorsAHSC AFP plan,
implement changes and monitor
progress
AHSC AFP review and report on results
The accountability cycle will ensure an effective feedback mechanism and continuous improvement of the AHSC AFP accountability process
50
3. Planning, Reporting, Monitoring Process
1. The template for reporting of measures will be finalized for each AHSC
2. The first year of AHSC accountability measures data will be populated by the ministry
3. Base line for measures based on historical data will be established for each AHSC
4. AHSC negotiations will include a discussion of planned future impacts on measures such as significant program changes
51
3. Planning, Reporting, Monitoring Process (cont’d)
5. Guidelines or performance corridors will be identified and discussed for each of the performance measures using baseline comparisons, comparable activity between AHSCs or standardized, recognized benchmarking
6. Results falling outside the performance corridors may trigger dialogue (AFP Governance/MOHLTC)
7. In future years, AHSC AFP Governance Organizations will populate the template based on planned programs and services
8. Semi-annual status reporting of data is recommended
9. The ministry will be responsible for administering the reporting process
52
4. Continue Developing a Comprehensive Set of AHSC AFP Accountability Measures
The current set of measures are foundational and based upon currently available data
This process would include work to collect primary data and develop required methodologies in support of the developmental measures that have been identified by the AWG
53
4. Continue Developing a Comprehensive Set of AHSC AFP Accountability Measures (cont’d)
The reporting of valid, reliable and high quality data that is externally verified is essential to support the AHSC AFP accountability process
Measures for the Leadership and Innovation domain should be a priority including: implementing and sharing new initiatives with respect to clinical care paths, practice guidelines and alternative models of care and testing and/or implementing internationally recognized best practices
54
5. Establish a Provincial Approach to the Next Level of AHSC AFP Accountability
The development of internal AHSC departmental or specialty level accountability measures
This phase should be accomplished using a provincial collaborative process to allow AHSCs to work together to share expertise for defining and developing benchmarks and best practices by specialty
For the departmental/specialty level of accountability it will be important to include direct participation of the participating physician groups by specialty using a common set of guidelines, a common mandate and an open, consultative process
55
6. Overarching AHSC AFP Planning Process
Develop a planning process that supports the accountability framework and takes into account and respects existing accountabilities of the parties to the AFP (physicians, hospitals, universities)
Will facilitate a stronger collaboration among the parties in terms of planning and accountability
56
Local AHSC AFPGovernance
Planning Parameters
AHSC AFP
Annual
Planning/Accountability
Process
Annual
Provincial AHSC AFP
Planning Parameters
Physician Annual Planning
University Annual
Planning
AHSC AFPAnnual
Accountability Cycle
Link to: AHSC AFP Agreements
Hospital Annual
Accountability(HAA)
University Annual
Accountability
Hospital Annual Planning(HAPS)
AFP GovernanceAnnual
Planning
57
Accountability Working Group Report
A draft of the Accountability Working Group report has been posted in the Working Documents section the AHSC AFP Task Force website and can be accessed via:www.health.gov.on.ca/login password: mohahsc
58
Thank you
QuestionsOr Comments
