College Ranking MetricsWhere Does UMKC Fit in the Rankings?

Why pay attention to rankings?

Although rankings are terribly imprecise, some of the metrics used are important

Prospective students pay attention and compare institutions on a number of indicators

UMKC can use metrics to benchmark against peer institutions and identify areas for improvement

Select National University and College Rankings

Undergraduate Education

U.S. News & World Report America’s Best Colleges 2009

http://colleges.usnews.rankingsandreviews.com/college

Forbes.com (The Center for College Affordability and Productivity) America’s Best Colleges 2008

http://www.forbes.com/2008/08/13/college-university-rankings-oped-college08-cx_rv_mn_0813intro.html

Research Productivity and Graduate Education

The Center for Measuring University Performance The Top American Research Universities

http://mup.asu.edu/research.html

National Research Council (New rankings due Spring 2009) Assessment of Research-Doctorate Programs (1995)

http://sites.nationalacademies.org/pga/Resdoc/index.htm

US News and World Report Rankings

Schools are categorized by mission derived from Carnegie classificationNational Universities

Offer full range of undergraduate majors, plus master’s and Ph.D. programs, and emphasize faculty research

262 institutions, including UMKC

Gather data from each college for up to 15 indicators of academic excellenceEach factor is assigned a weight that reflects how

much it mattersColleges are ranked against their peers based on

their composite weighted score

US News Tier Structure

Top Schools Institutions ranked 1 to 130

Tier 3 Institutions ranked 134 to 188

UMKC: Ranked 165th

Tier 4 Institutions ranked 197 to 260

2009 U.S. News MethodologyMeasures and Their Weights

Student: Faculty Ratio

Proportion of FT Faculty

Acceptance Rate

Class Size: More than 50

Faculty w/ Terminal Degree

Freshman Retention Rate

Grad Rate Performance

Alumni Giving

Class Size: Fewer than 20

Freshmen in Top 10%

Faculty Compensation

SAT/ACT Score

Expenditures per FTE Student

Six-year Graduation Rate

Peer Assessment

0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0%

1.0%

1.0%

1.5%

2.0%

3.0%

4.0%

5.0%

5.0%

6.0%

6.0%

7.0%

7.5%

10.0%

16.0%

25.0%

U.S. News Measures by Cluster

Peer Assessment

25%

Selectivity• SAT/ACT• Top 10%• Acceptanc

e

15%

Faculty Resources

Graduation & Retention

• Class Size• Salaries• Faculty• Student/Faculty

• Freshman Persistence

• Graduation• Grad Rate

Performance

20%

25%

Alumni Giving

5%

Financial Resources

10%

Issues with U.S. News Metrics• Peer assessment: reputational only, heaviest weight

• Graduation/retention rates: insensitive to different populations; reported only for limited subset of students (first-time, full-time freshmen)

• Faculty resources: class sizes used not necessarily ideal; model biased towards small colleges

• Student selectivity: ACT/SAT scores at entrance are input variables, not a reflection of institutional performance

• Financial resources: educational expenditures per student are difficult to compare across institutions

• Alumni Giving: a proxy for student satisfaction with their alma mater; only counts undergraduate giving over the past two years, and only counts former students who received a degree• Best for small, homogeneous liberal arts colleges (i.e., Ivy League Universities

and Private Liberal Arts Colleges)

Where Does UMKC Rank?UM-Kansas City 2007 Edition 2008 Edition 2009 Edition Weighting Tier Increase 1%

Overall Rank 175th 172nd 165th   3rd  Overall score 27 23 23

Peer Assessment Rank   176th 158th   3rd  Peer assessment score 2.5 2.5 2.5 25.0%

Graduation & Retention Rank 192nd 206th 208th   4th  Avg. freshman retention rate 73% 72% 72% 4.0% 9 studentsSix-year graduation rate 47% 45% 43% 16.0% 7 students

Graduation Rate Performance Rank   249th 250th 5.0% 4th   Predicted 65% 63% 61% Actual 47% 45% 43% Difference -18% -18% -18%

Faculty Resources Rank 131st 116th 90th   1st  Classes under 20 students 53% 56% 54% 6.0% 12 sectionsClasses of 50 or more students 7% 7% 6% 2.0% 11 sectionsStudent/faculty ratio 11/1 13/1 9/1 1.0%Faculty who are full-time 82% 83% 84% 1.0% 27 positionsFull-time faculty with terminal degree 83% 84% 3.0% 10 FacultyFaculty Salaries 7.0%

Selectivity Rank 106th 98th 99th   1st  ACT 25th-75th percentile 21-27 20-27 21-28 7.5%Freshman in top 10% of HS class 30% 34% 34% 6.0% 8 studentsAcceptance rate 75% 71% 60% 1.5% 34 students

Financial Resources Rank 131st 132nd 137th 10.0% 3rd  

Alumni Giving Rank 229th 233rd 223rd   4th  Avg. alumni giving rate 5% 5% 6% 5.0% 435 alumni

UMKC’s Operational PeersGeorge Univ. of Univ. of Virginia

UMKC Mason Univ. UAB Louisville South Florida Comm. Univ.

Overall Rank 165th 149th 145th 170th 170th 170th

Peer Assessment Score 2.5 2.9 2.7 2.6 2.5 2.7

Graduation & Retention Rank 208th 151st 230th 223rd 185th 197thAvg. freshman persistence rate 72% 85% 76% 77% 81% 81%Six-year graduation rate 43% 59% 38% 44% 49% 47%Graduation rate performance Predicted 61% 56% 53% 56% 52% 52% Actual 43% 59% 38% 44% 49% 47% Difference -18% 3% -15% -12% -3% -5%

Faculty Resources Rank 90th 203rd 115th 211th 234th 143rdClasses under 20 students 54% 34% 38% 25% 29% 44%Classes of 50 or more students 6% 13% 13% 13% 14% 11%Student/faculty ratio 9/1 15/1 17/1 18/1 19/1 18/1Faculty who are full-time 84% 78% 97% 84% 97% 84%Full-time faculty with terminal degree 84% 91% 86% 89% 80%

Selectivity Rank 99th 179th 111th 122nd 133rd 209thACT 25th-75th percentile 21-28 1020-1210 21-27 21-27 1010-1210 960-1170Freshmen in top 10% of HS class 34% 18% 30% 24% 27% 15%Acceptance rate 60% 56% 77% 70% 50% 62%

Financial Resources Rank 137th 197th 22nd 72nd 169th 158th

Alumni Giving Rank 223rd 190th 148th 118th 148th 182ndAvg. alumni giving rate 6% 8% 11% 13% 11% 9%

UMKC’s Aspirational PeersUniv. of SUNY at Temple Univ. of Univ. of

UMKC Pittsburgh Buffalo University Illinois-Chicago Cincinnati

Overall Rank 165th 58th 121st 134th 134th 149th

Peer Assessment Score 2.5 3.4 2.9 2.8 3.0 2.6

Graduation & Retention Rank 208th 74th 125th 135th 188th 181stAvg. freshman persistence rate 72% 90% 87% 85% 78% 80%Six-year graduation rate 43% 75% 63% 59% 52% 50%Graduation rate performance Predicted 61% 67% 59% 51% 58% 51% Actual 43% 75% 63% 59% 50% 52% Difference -18% 8% 4% 8% -8% 1%

Faculty Resources Rank 90th 107th 149th 178th 217th 186thClasses under 20 students 54% 43% 35% 42% 37% 47%Classes of 50 or more students 6% 15% 22% 5% 19% 8%Student/faculty ratio 9/1 16/1 16/1 17/1 16/1 14/1Faculty who are full-time 84% 92% 85% 73% 91% 99%Full-time faculty with terminal degree 84% 92% 97% 70% 78% 61%

Selectivity Rank 99th 67th 128th 173rd 118th 128thACT 25th-75th percentile 21-28 1150-1340 1040-1260 980-1180 21-26 21-27Freshmen in top 10% of HS class 34% 48% 24% 19% 23% 20%Acceptance rate 60% 56% 52% 63% 64% 75%

Financial Resources Rank 137th 35th 105th 144th 34th 79th

Alumni Giving Rank 223rd 95th 174th 166th 214 182ndAvg. alumni giving rate 6% 15% 9% 10% 7% 8%

UMKC’s Peer Institutions(as identified by UM System)

• Cleveland State University*• East Tennessee State University• George Mason University• Georgia State University*• Indiana Univ. - Purdue

Univ./Indy*• SUNY at Buffalo• Temple University*• University of Akron• University of Alabama -

Birmingham*• University of Arkansas at Little

Rock• University of Cincinnati*• University of Houston -

University Park*

• University of Illinois – Chicago*

• University of Louisville• University of Memphis*• University of Nevada - Las

Vegas• University of Pittsburgh*• University of South Alabama• University of South Florida• University of Toledo*• University of Wisconsin -

Milwaukee*• Virginia Commonwealth

University*• Wayne State University*• Wright State University* Members of the Urban 21

Where Can Improvement be Made?

Measures driven by single year data

• Peer Assessment• SAT/ACT Scores• Top 10% Freshmen• Class Size• Full-time Faculty• Faculty with Terminal Degrees• Acceptance Rate• Student/Faculty Ratio• Graduation Rate Performance

Measures driven by multiple-year data

• Expenditures per Student (2 yr avg.)

• Faculty Compensation (2 yr avg.)

• Alumni Giving (2 yr avg.)• Freshman Persistence (4 yr

avg.)• Graduation Rate (4 yr avg.)

Forbes.com Rating Factors

25.0%

25.0%16.7%

16.7%16.7%

Who's Who AlumniStudent EvaluationsFour-Year Graduation RateNational AwardsAccumulated Student Debt

Issues with Forbes.com Ratings• 569 schools ranked; every different type of four year institution is

ranked together (apples and oranges)

• RateMyProfessors.com as a measure of student learning?

• Who’s Who in America used as a dubious measure of the achievement of graduates (and as a proxy for academic quality)

• Ranking schools based on accumulated debt – also a proxy for the net worth of the students attending the institution

• Only one school in the top 50 has an enrollment over 10,000 (average N of top 50=2,285)

• “Forbes methodology … reveals a profound bias in favor of wealth and class and against institutions that have distinctive missions – especially those that serve low income students and significant populations of minority students.” (Patricia McGuire, Inside Higher Education, August 28, 2008)

UMKC’s Operational PeersGeorge Univ. of Univ. of Virginia

Weighting UMKC Mason Univ. UAB Louisville South Florida Comm. Univ.

Overall Rank   539th 382nd 354th 400th 337th 433rd

Who's Who Rank 25.0% 188th 355th 482nd 486th 372nd 436thWho's Who Score 0.004324 0.002129 0.001036 0.001024 0.001887 0.001360

Rate My Professor Rank 25.0% 352nd 368th 214th 322nd 322nd 322ndOverall 3.630 3.700 3.810 3.710 3.750 3.730Ease 3.000 3.230 3.270 3.190 3.310 3.250RMP Score 7.890 7.870 8.160 7.940 7.940 7.940

Student National Awards Rank 8.3% 335th 222nd 143rd 317th 334th 251stStudent Awards Score Not Reported 0.000530 0.001087 0.000079 0.000037 0.000345

Faculty National Awards Rank 8.3% 84th 124th 151st 151st 141st 147thFaculty Awards Score 0.002188 0.000939 Not Reported Not Reported 0.000545 0.000349

Average Student Debt Rank 16.7% 555th 113th 175th 48th 15th 239thAverage Student Debt 29,843 12,982 14,222 11,302 9,770 15,345

Graduation Rate Rank 16.7% 538th 481st 546th 549th 506th 486thFour Year Graduation Rate Predicted 28% 33% 27% 26% 31% 27% Actual 16% 26% 14% 13% 22% 21% Difference -12% -7% -13% -13% -9% -6%

UMKC’s Aspirational PeersUniv. of SUNY at Temple Univ. of Univ. of

Weighting UMKC Pittsburgh Buffalo University Illinois-Chicago Cincinnati

Overall Rank   539th 298th 436th 546th 547th 516th

Who's Who Rank 25.0% 188th 273rd 194th 283rd 463rd 366thWho's Who Score 0.004324 0.002991 0.004238 0.002816 0.001203 0.002008

Rate My Professor Rank 25.0% 352nd 403rd 493rd 505th 429th 503rdOverall 3.630 3.489 3.621 3.600 3.630 3.590Ease 3.000 3.007 3.220 3.190 3.120 3.150RMP Score 7.890 7.460 7.643 7.610 7.770 7.620

Student National Awards Rank 8.3% 335th 91st 250th 258th 217th 275thStudent Awards Score Not Reported 0.002287 0.000346 0.000318 0.000563 0.000232

Faculty National Awards Rank 8.3% 84th 108th 151st 93rd 146th 151stFaculty Awards Score 0.002188 0.001290 Not Reported 0.001811 0.001811 Not Reported

Average Student Debt Rank 16.7% 555th 166th 244th 495th 436th 249thAverage Student Debt 29,843 14,111 15,452 23,171 20,311 15,535

Graduation Rate Rank 16.7% 538th 282nd 442nd 431st 531st 504thFour Year Graduation Rate Predicted 28% 42% 41% 29% 32% 23% Actual 16% 46% 35% 27% 20% 16% Difference -12% 4% -6% -2% -12% -7%

The Center for Measuring University Performance Rankings

• Determines the Top American Research Universities by their rank on nine different measures: ▫ Total Research Expenditures▫ Federal Research Expenditures▫ Endowment Assets ▫ Annual Giving ▫ National Academy Membership

National Academy of Sciences, National Academy of Engineering, and Institute of Medicine membership directories

▫ Faculty Awards▫ Fulbrights, Guggenheims, National endowment for the Humanities Awards,

NSF CAREER Awards, Sloan Fellowships, etc.

▫ Doctorates Granted▫ Postdoctoral Appointees▫ SAT/ACT range

Issues with The Center Metrics• Focused on research productivity and prestige, designed to assess

research excellence, not educational excellence, so metrics focused on research productivity of faculty, not educational programs

• Funded research, especially federally funded research, seen as most prestigious; other types of scholarship and creative activity not as valued

• Some emphasis on faculty awards compensates for heavy emphasis on funded research

• Overall rankings reflect numbers of faculty and types of programs – Larger Universities with Medical Schools, Engineering Schools, and Agricultural Schools do best (i.e., large land grant Universities)

• These rankings not concerned with Universities with other missions (e.g., urban serving institutions)

UMKC’s Operational PeersGeorge Univ. of Univ. of Virginia

UMKC Mason Univ. UAB Louisville South Florida Comm. Univ.

Total Research Rank 150th 131st 26th 72nd 40th 67thTotal Research x1000 30,780 42,630 318,536 128,347 259,218 146,105

Federal Research Rank 149th 110th 15th 76th 40th 60thFederal Research x1000 16,960 36,475 288,434 67,104 142,580 97,322

Endowment Assets Rank 115th N/A 60th 28th 66th 74thEndowment Assets x1000 135,000 N/A 346,330 680,251 329,832 265,653

Annual Giving Rank 104th 105th 37th 52nd 70th 50thAnnual Giving x1000 23,358 23,283 75,277 61,142 46,347 65,107

National Academy Members Rank 117th 72nd 42nd 72nd 72nd 58thNational Academy Members 0 2 8 2 2 4

Faculty Awards Rank 265th 50th 65th 72nd 42nd 72ndFaculty Awards 0 8 6 5 10 5

Doctorates Granted Rank 126th 68th 79th 77th 57th 54thDoctorates Granted 61 163 143 144 184 197

Post Doctoral Appointees Rank 103rd 182nd 41st 60th 50th 42ndPost Doctoral Appointees 41 0 232 136 179 223

SAT or ACT Rank N/A 128th 154th 105th 99th 195thSAT range or ACT 21-27 490-600; 510-610 20-26 21-26 510-600; 520-610 480-590; 470-580

UMKC’s Aspirational PeersUniv. of SUNY at Temple Univ. of Univ. of

UMKC Pittsburgh Buffalo University Illinois-Chicago Cincinnati

Total Research Rank 150th 14th 36th 100th 27th 33rdTotal Research x1000 30,780 510,943 267,271 83,062 318,279 286,036

Federal Research Rank 149th 7th 56th 89th 28th 25thFederal Research x1000 16,960 420,281 151,890 53,987 198,163 202,654

Endowment Assets Rank 115th 9th 40th 81st 102nd 18thEndowment Assets x1000 135,000 1,802,859 499,632 204,178 162,798 1,101,100

Annual Giving Rank 104th 21st 96th 74th 73rd 41stAnnual Giving x1000 23,358 116,551 28,462 41,883 43,780 70,706

National Academy Members Rank 117th 23rd 72nd 117th 58th 53rdNational Academy Members 0 25 2 0 4 6

Faculty Awards Rank 265th 13th 42nd 87th 30th 30thFaculty Awards 0 23 10 4 12 12

Doctorates Granted Rank 126th 18th 31st 24th 37th 45thDoctorates Granted 61 412 353 383 311 264

Post Doctoral Appointees Rank 103rd 6th 24th 84th 39th 30thPost Doctoral Appointees 41 722 328 75 235 276

SAT or ACT Rank N/A 24th 273rd 138th 154th 154thSAT range or ACT 21-27 560-660; 580-670 520-610; 550-640 490-590; 500-600 20-26 20-26

UM System Accountability Metrics• UM System metrics found within various

rankings:▫ US News Rankings

Freshmen persistence Rate six-year graduation rate Percent of tenured and tenure track faculty with terminal degrees Number of entering students in the top 10% of high school graduation class Ranked faculty salaries compared to peers and market Faculty and staff benefits compared to peers and market Freshman applicant acceptance rate Educational expenditures per FTE student Alumni giving percent

▫ Forbes.com Rankings Number of student awards Number of faculty awards Average undergraduate debt burden upon graduation Four-year graduation rate

▫ The Center Rankings Total externally sponsored research expenditures Total federally funded federal research expenditures Annual fundraising Number of faculty awards, fellowships and memberships Total degrees awarded by level (including doctorate)