Upload
dale-holway
View
214
Download
1
Embed Size (px)
Citation preview
Research Assessment and Research Assessment and Funding: the experience of other Funding: the experience of other
countriescountries
Dr Lisa Lucas
Graduate School of Education
University of Bristol
Change in RAE gradingsChange in RAE gradings% of top grades (5 & 5*) from 1992 to % of top grades (5 & 5*) from 1992 to
2001 by subject area2001 by subject area
22%(18)
54%(41)
9%11%(9)
46%(41)
10%15%(10)
38%(18)
9%
0%
10%
20%
30%
40%
50%
60%
RAE 1992 RAE 1996 RAE 2001
Biology
English
Sociology
The Research Game in Academic Life The Research Game in Academic Life ((Lucas, 2006Lucas, 2006))
Intensification of the management and organisation of research activities
Differentiation of academics within departments over status/workloads
Struggles over classification of research active/research inactive
Lack of value perceived for teaching and associated work
Prioritising of research areas that can attract high levels of research funding and would be worthy of submission to high ranking journals
UK University Core Funding for Research UK University Core Funding for Research and Teaching 2005-6and Teaching 2005-6
Source: Higher Education Funding Council (HEFCE)Source: Higher Education Funding Council (HEFCE)
Institution Core Funds
Teaching (£M)
Core Funds
Research (£M)
Ratio
(% research)
Cambridge 57.36 92.37 62%
UCL 58.05 92.98 62%
Oxford 57.72 90.16 61%
Manchester 83.38 68.93 45%
Bristol 53.22 37.86 42%
Kingston 45.44 0.81 2%
Coventry 36.54 0.53 1%
Focus of PresentationFocus of Presentation
What can be learned from other national systems of funding and evaluating research in universities that utilise metrics?
Can other national systems provide alternative ideas to inform the development of the UK system of funding and evaluating research?
University State Funding and the Higher University State Funding and the Higher Education Landscape in the 21Education Landscape in the 21stst Century Century
Reduction in State funding to universities– Resource Dependency Theory (Slaughter & Leslie,
1997)
Global Competition– Creating ‘World Class’ Universities (global
league tables)
Globalisation and Higher Education– Convergence of Higher Education Policies?
Measuring the Scientific Impact of Measuring the Scientific Impact of Nations?Nations?
Rank Country Top 1% of highly cited publications
1997-2001
1 US 23,723
2 EU15 14,099
3 UK 4,831
7 Canada 2,195
10 Netherlands 1,435
11 Australia 1,049
20 China 375
Source: King, D.A. (2004) The Scientific Impact of Nations, Nature,
Vol 430, 15th July.
National Case Studies of Research National Case Studies of Research Funding and EvaluationFunding and Evaluation
Hong KongThe NetherlandsAustralia and New Zealand
Higher Education System in Hong KongHigher Education System in Hong Kong
Eight institutions Funded by the University Grants Committee
(UGC)“As far as the UGC is concerned, the UGC advocates
role differentiation among institutions, and the concentration of resources to reward performance
and encourage the growth of centres of excellence.” (UGC, Facts and Figures, 2002)
Hong Kong RAE 2006Hong Kong RAE 2006
Based originally on the UK RAESingle quality threshold at attainable level
of excellence, no gradingCarnegie Classifications:
– scholarship of discovery – Scholarship of integration – Scholarship of application – Scholarship of teaching
Hong Kong System: key issuesHong Kong System: key issues
Single quality threshold – no differentiation between research ‘active’ and ‘inactive’ staff.
Allowing for multiple dimensions of excellence by broadening the definition of ‘research’ to forms of scholarship.
BUT– Role differentiation amongst institutions mandated.– Greater compulsion for EVERYONE to do research.– Multiple dimensions of evaluation less easy in practice.– Pressure to publish ‘internationally’.
Higher Education System in the Higher Education System in the NetherlandsNetherlands
“The universities prepare students for independent scientific work in an academic or professional setting and the hogescholen prepares students to practise a profession
and enable them to function self-consciously in society at large.”
13 universities and 50 hogescholen– Source: Boezerooy, P (2003) Higher Education in the
Netherlands: Country Report, CHEPS
Research Funding and Evaluation in the Research Funding and Evaluation in the Netherlands Netherlands (VSNU)(VSNU)
Universities are funded for teaching and research as a block grant based primarily on historical circumstances
Development of Research Institute and Schools Structure (separate within Faculties)
Research (mainly self-evaluation) based on:– Quality– Productivity– Relevance– Vitality and Feasibility
The Dutch System: key issuesThe Dutch System: key issues
Funding not linked to evaluation of research work Self-evaluation of research Research School/Institute structure means that
academics can be either research or teaching only BUT
– Binary system/small university sector– Institutions use evaluation to manage, control and direct
research activities– Possible separation of teaching and research staff within
faculties but also more collaboration of staff across institutions
Research Funding and Evaluation in Research Funding and Evaluation in Australia (2008?)Australia (2008?)
40 universities Imminent change to the Research Quality
Framework (RQF) – combining quantitative and qualitative evaluation
On going debates on the formation of the RQF Concern with ‘basket of metric indicators’ –
metrics working group Concern with Impact of research (social, economic,
environmental) – impact working group
Distribution of publication output by field, Distribution of publication output by field, Australian universities, 1999-2001 Australian universities, 1999-2001 (Butler, 2006)(Butler, 2006)
Field Books Book chapter
Journal article
Conf. paper
Band 1 Chem. Sc.
0.2 2.1 95.7 1.9
Band 2 Psych. 1.5 17.4 76.2 4.9
Band 3 Econ. 2.9 24.5 64.5 8.0
Band 4 Politics 5.8 37.3 46.1 10.8
Research Funding and Evaluation in Research Funding and Evaluation in New Zealand (2006, 2012)New Zealand (2006, 2012)
Performance-based Research Funding (PBRF) – quantitative and qualitative indicators– Quality of researchers (60%)– Reflect research degree completions (25%)– Reflect external income (15%)
45 institutions Some Key Aims of the PBRF
– Ensure that research continues to support degree and postgraduate teaching
– Prevent undue concentration of funding that would undermine research support for all degrees
What can be learned from other national What can be learned from other national systems that utilise metrics?systems that utilise metrics?
Quantitative measures should inform qualitative judgement not replace it.
A ‘basket of metrics’ needs to be utilised rather than simplistic single indicators.
Perverse research practices can be encouraged where simplistic measures are utilised (Butler, 2003).
Can other national systems provide Can other national systems provide alternative ideas?alternative ideas?
Widening the definition of research and encouraging the integration of research and teaching
Encouraging greater collaboration across the sector to build capacity
Research for whom? Impact and communication of research Importance of combined quantitative and qualitative
indicators and variety of indicators to be utilised Indicator utilised that are appropriate for different subject
areas Holistic approach – concern to support the whole higher
education sector
Dr Martin Dr Martin (Biology, Golden County University)(Biology, Golden County University)
“I would rather see all the vice chancellors lined up for a hundred yards dash and just assign money
on that basis, because that exercise would take at the most two minutes. Even the weakest vice chancellor could do a hundred yards in two
minutes and then get on with life. It is about as rational as that. At least you could train your
vice chancellor and pick a healthy one. At least you would have a use for a vice chancellor at long last, you’d be able to select on a rational basis. It might televise well and you might get money from the rights on watching it. And you
might get rid of a few each time.”
ReferencesReferences
Butler, L. (2003) Explaining Australia’s increased share of ISI publications – the effects of funding formula based on publication counts, Research Policy, 32: 143-155.
Butler, L. (2006) Research Assessment: moving beyond journal outputs, SPRU 40th Anniversary Conference – The Future of Science, Technology and Innovation Policy, University of Sussex, 11-13 September.
DEST, Research Quality Framework, Australia http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issues/research_quality_framework/default.htm
King, D.A. (2004) The Scientific Impact of Nations, Nature, 430 (15th July): 311-6. Lucas, L. (2006) The Research Game in Academic Life, Maidenhead: SRHE/Open
University Press Slaughter, S. & Leslie, L. (1997) Academic Capitalism: politics, policies and the
entrepreneurial university, Baltimore: The John Hopkins University Press. Tertiary Education Comission, Performance-based Research Fund, New Zealand,
http://www.tec.govt.nz/funding/research/pbrf/pbrf.htm UGC (2002) Facts and Figures, Hong Kong SAR.