Upload
shana-conley
View
217
Download
2
Tags:
Embed Size (px)
Citation preview
PERFORMANCE MEASUREMENT: WHAT MIGHT BE AHEAD?
Chicago, Illinois
April 26, 2004
Accounting Principles Council Strategic Business Reporting Project
Project Team
Jennifer Taylor, NMSU, Chair
Tammy Anthony, NMSU
Teresa Gordon, University of Idaho
Roger Patterson, University of North Carolina-Chapel Hill
Carolann Sagesse, Cornell University
Sue Menditto, NACUBO
Kim Dight, NACUBO
Why is NACUBO’s Accounting Principles Council concerned with Performance Measurement?
Both GASB and FASB have implemented long-term projects to address the issue.
Accounting Principles Council Strategic Business Reporting Project
Accounting Principles Council Strategic Business Reporting Project
Performance Measurement Background
– GASB research effort to enhance service efforts and accomplishments reporting (SEA)
– SEA is non-financial performance information for governmental organizations
– SEA is rooted in GASB Concept Statement No.1, “Objectives of Financial Reporting”
• Assist in fulfilling government’s duty to be publicly accountable
• Enable users to assess accountability
Performance Measurement Background
– GASB Concepts statement No. 2, “Service Efforts and Accomplishments Reporting”
– Conceptually links accountability and external financial reporting
Accounting Principles Council Strategic Business Reporting Project
Performance Measurement Background
– GASB received Sloan Foundation research grants in 1997 and 2000 • To expand performance measurement
research• To understand external reporting of
performance information– The grants have six research phases
Accounting Principles Council Strategic Business Reporting Project
Six Phases of GASB Research Project
1. Performance measurement website
2. Case studies based on interviews and surveys on uses and effects of using performance measures
3. Citizen discussion groups that evaluate users responses to performance measures
4. Develop suggested criteria to enable performance information reporting
5. Experimentation with suggested criteria (future)
6. Evaluate effectiveness of suggested criteria (future)
Accounting Principles Council Strategic Business Reporting Project
NACUBO Performance Measurement Project Goals
• Address GASB performance reporting suggested criteria and related issues (GASB phase 4 research results)
• Inform GASB of and proactively represent higher education performance measurement practices (NACUBO advocacy role)
• Develop a simple, visual template for reporting meaningful performance information both internally and externally (GASB phase 5 – experimentation with criteria and external performance reporting)
• Develop a model useful to both public and private institutions (NACUBO Industry Guidance)
Accounting Principles Council Strategic Business Reporting Project
Project Approach
• Worked with GASB for guidance • Developed a survey
– To determine if performance reporting currently exists– To determine measures considered important and
relevant for higher education– To assess opinions and relevance of the GASB 16
criteria
Accounting Principles Council Strategic Business Reporting Project
Accounting Principles Council Strategic Business Reporting Project
Project Timeline
• Conduct Performance Measurement Survey – January 2004
• Analyze Survey – February 2004• Review Sample of Existing PM Reports and Prior
Research Results – April 2004 • Review Draft Template Content and Format with
Focus Groups – Summer 2004
• Develop Template – Winter 2004
Accounting Principles Council Strategic Business Reporting Project
Survey Goals
• Identify prevalence and type of PM reporting currently in place within higher education (industry-wide and by constituent group)
• Identify commonly valued KPI at both public and private institutions
• Assess importance of non-financial indicators in higher education
• Provide GASB with input:– Opinions about 16 reporting criteria– Use of outcome-driven and efficiency-based
measures
What is a KPI?
KPI = Key Performance IndicatorTypes of Performance Measures:• Input – selectivity or enrollment stats• Output – graduation stats, national exam• Outcome – perceptions surveys, employment
stats• Efficiency – cost per student• Effectiveness (Productivity) – difference in
entrance and exit scores
Accounting Principles Council Strategic Business Reporting Project
GASB Performance Reporting Criteria
1. Purpose/scope is stated clearly
2. Institutional goals/objectives stated clearly
3. Citizen involvement in goal setting
4. Multiple levels of detail presented
5. Qualitative analysis of results/challenges
6. Concise yet comprehensive, focus on key issues
7. Assessment of information reliability
8. Measures are relevant, link to goals
Accounting Principles Council Strategic Business Reporting Project
GASB Performance Reporting Criteria
9. Measures should relate to cost/resources/efficiency
10. Report citizen perceptions of quality/results
11. Present comparative information
12. Discuss factors affecting results
13. Show aggregation/disaggregation by user group
14. Measures should be consistent across periods, or explain changes
15. Easy to find, access, understand
16. Available annually and in timely manner
• http://www.seagov.org/sea_gasb_project/criteria_summary.pdf
Accounting Principles Council Strategic Business Reporting Project
Accounting Principles Council Strategic Business Reporting Project
Survey Highlights
• Surveyed all NACUBO Primary Rep’s• Response rate: 12%• Survey supports continuing template development
– Majority of respondents do some type of reporting, large percentage of these include internal reporting
– Majority do not have mandated reporting format– Majority do not have external guidance for voluntary
reporting
• External reports are primarily extracts of internal reports
Accounting Principles Council Strategic Business Reporting Project
Independents Publics
Report KPIs Externally
38% 62%
Use KPIs Internally
46% 54%
KPIs Reported and Used
Accounting Principles Council Strategic Business Reporting Project
Survey Highlights
• Roughly half of respondents relate reports to audited financial statements
• Top priority KPI’s identified• Certain KPI’s indicated as not being appropriate to
report• A wider variety of measures are used internally than
externally• A different approach to internal reporting• What is “important” versus what is “recommended”
to be included are not always the same KPI’s!
Survey Highlights
Independents vs. Publics• Independents report/use selectivity measures
(externally/internally) more often than publics• Publics report persistence toward graduation
statistics externally more often as compared to independents
• But both USE persistence statistics internally at comparable levels
Accounting Principles Council Strategic Business Reporting Project
Included in External Reports
Number of respondents: 159
Number Percentage
Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours)
146 91.8%
Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years)
120 75.5%
Efficiency or comparative financial data/ratios 110 69.2%
Graduation statistics (e.g., degree granted by level or field of study)
101 63.5%
Diversity measures (e.g., statistics on race or gender for students, faculty or staff)
100 62.9%
Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility)
94 59.1%
Selectivity measures (e.g., acceptance rate, high school GPA, SAT or ACT scores)
90 56.6%
Used Internally by Management
Number of respondents: 156
Number Percentage
Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours)
156 100.0%
Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years)
137 87.8%
Graduation statistics (e.g., degree granted by level or field of study)
135 86.5%
Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility)
130 83.3%
Efficiency or comparative financial data/ratios 126 80.8%
Diversity measures (e.g., statistics on race or gender for students, faculty or staff)
121 77.6%
Student satisfaction or graduating senior survey results
118 75.6%
Accounting Principles Council Strategic Business Reporting Project
What SHOULD Be Reported
• Externally – top five were selected by over 67% of respondents
• Internally – top six were selected by 95% or more of respondents– Most of the indicators were identified as useful
for internal management purposes
Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours) 88%Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years) 83%Graduation statistics (e.g., degree granted by level or field of study) 81%Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility) 80%Diversity measures (e.g., statistics on race or gender for students, faculty or staff) 67%
Top 5 Measures that SHOULD be Reported Externally
Accounting Principles Council Strategic Business Reporting Project
Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years) 98%
Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours) 97%
Efficiency or comparative financial data/ratios 97%Graduation statistics (e.g., degree granted by level or field of study) 95%Student satisfaction or graduating senior survey results 95%Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility) 95%
Top 5 Measures that SHOULD be Used Internally
Accounting Principles Council Strategic Business Reporting Project
Accounting Principles Council Strategic Business Reporting Project
Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years)
Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours)Graduation statistics (e.g., degree granted by level or field of study)Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility)
Four Measures Common to External and Internal Reporting
Enrollment and Persistence Measures
• Top the list on what is being reported externally and what should be reported externally
• Top the list on what is being used internally and what should be reported internally
Accounting Principles Council Strategic Business Reporting Project
Measures which Should NOT Be Reported Externally or Internally
Faculty or staff morale, comparative salary studies 15%
Selectivity measures (e.g., acceptance rate, high school GPA, SAT or ACT scores) 13%
Performance on nationally recognized exams 13%Alumni or employer survey results 8%
Accounting Principles Council Strategic Business Reporting Project
GASB Reporting Criteria – rating (1-7)
Accounting Principles Council Strategic Business Reporting Project
All Respondents
Respondents Reporting Externally
Consistency from Year to Year 5.18 5.13
Regular and Timely Reporting 5.06 5.01
Relevant Measures of Results 5.02 4.95
Reliable Information 4.89 4.82
Focus on Key Measures 4.82 4.79
Mission Statement and Major Goals 4.76 4.78
Survey results tabulated.
What’s next
External research and development of a reporting template.
Accounting Principles Council Strategic Business Reporting Project
Accounting Principles Council Strategic Business Reporting Project
Objectives of Reporting Template• Verify external reporting versus survey responses• Identify additional categories of indicators
• Find examples of indicators Approach to Compiling Reporting Template• Approximately 15% of survey respondents
provided links to external reporting• Researched institutions not responding to survey
who provide external reporting on their website• Tallied indicators
Research Highlights
• 45 institutional reports reviewed • 23 categories of KPI’s identified
– 50% not on 2004 survey• 272 individual KPI’s identified
– 17% in Common Data Set or IPEDS– 13% recommended from prior surveys/studies
Accounting Principles Council Strategic Business Reporting Project
Accounting Principles Council Strategic Business Reporting Project
Research Highlights
Most Common Measurement CategoriesGraduation StatisticsPersistence and Graduation OutcomesResearch MeasuresEnrollment StatisticsSelectivity MeasuresDiversity Measures
Research Highlights
Most Common Measures• Freshman retention rate• Percent graduating in 4-6 years• Degrees granted by level or field of study• External dollars received or expended through
research/creative activity• Undergraduate transfer data• Annual giving• FTE and Headcount• Race/gender/nationality of students/faculty
Accounting Principles Council Strategic Business Reporting Project
Research Highlights
Most Common Weighted Measures• Freshman retention rate• Percent graduating in 4-6 years• Degrees granted by level or field of study• Entering test scores (ACT, SAT) • FTE and Headcount• Number of minority enrollments/ratio• Student/faculty ratio
Accounting Principles Council Strategic Business Reporting Project
Research Highlights
Most Common Unweighted Measures• External dollars received or expended through
research• Average class size• Comparative exam results• Total number of new students enrolled each Fall • Perceived quality of degree• Functional expense as % of total expense• Number of sponsored awards/national ranking
Accounting Principles Council Strategic Business Reporting Project
Accounting Principles Council Strategic Business Reporting Project
Reporting template complete.
What’s next
Focus Groups
Objectives of Focus Groups
• Develop relevant new questions not addressed by survey
• Consider next steps – Should the project aim for assistance by
conducting experimentation as set forth in the GASB Service Efforts and Accomplishments Reporting Program?
– Reaction to draft template
Accounting Principles Council Strategic Business Reporting Project
Focus Groups Planned
• Tuesday, April 27 at 9:30 am• NACUBO Integrated Planning and Budgeting
Workshop - May• Higher Education Association Community (ACE,
AIR, SHEEO, SHEFO, etc.) Summer• NACUBO Annual Meeting - July
Accounting Principles Council Strategic Business Reporting Project