View
214
Download
1
Category
Tags:
Preview:
Citation preview
A review of the Value for Money assessment of Science Foundation Ireland
Ali BasakRichard O'Carroll
MSc EPS Year 2EC8011 – Assignment 2
Background
Science Foundation Ireland (SFI) invests in academic researchers and research teams who are most likely to generate new knowledge, leading edge technologies and competitive enterprises in the fields of science and engineering underpinning three broad areas: Biotechnology, Information and communications technology, Sustainable energy and energy-efficient technologies
Activities previously reviewed by Brook in 2005
This evaluation completed by Indecon in 2008
Assessment takes place in context of the Government’s Value for Money and Policy Review Initiative and looks at the SFI’s
programmes under three headings:
Terms of reference
Objectives
Impacts
Complentarities/Linkages
1. Examine validity of programmes in meeting the longterm objectives of SPI to create highly visible critical mass of word-class rearch excellence
2. Examine validity of programmes in context of changing circumstances and overall consistency with overall objectives of the SSTI, other relevant policies and EU strategies in the area
3. Determine probable outcomes that could result from discontinuation, reduction or expansion of any programme
4. Review individual objectives of each programme and assess extent to which they are being met
5. Examine separate programmes in terms of comprehensive measurement of inputs and outputs
1. Review effectiveness of operation of individual programmes in terms of outputs and impacts where possible. Also quality of programmes and possible alternatives to them.
2. Review user satisfaction with management and implementation of programmes by SFI
1. Review effectiveness of operation of individual programmes in terms of outputs and impacts where possible. Also quality of programmes and possible alternatives to them2. Review user satisfaction with management and implementation of programmes by SFI
Methodology
Evaluation of programme outputs
Review of research outputs
Review ofCollaboration
Activity
Review ofCommercialisation
Activity
Outputs of journal-based peer review articles published by SFI funded researchers
Outputs in relation to international conference presentations or papers delivered by SFI-funded researchersInternational collaborations
Industry collaborations
Views of industry on SFI objectives
Data considered: SFI commitments & expenditure; Research activities & outputs; Consultation of stakeholders/experts & surveys
Programme effectiveness & impact
Assessment of wider economic impacts
Programme Management & Monitoring
International Comparative
Position of SFI
Assessment of research outputs
& quality
Early citation rates
“International” quality of outputs based on appearance in peer-reviewed journals
“Productivity" of researchers
Programme effectiveness & impact
Assessment of wider economic impacts
Programme Management & Monitoring
International Comparative
Position of SFI
Assessment of research outputs
& quality
Economic benefits of investment in human capital
Inward Investment effects – based on discussions with the IDA and a reference to three partnerships between multinationals and Irish third level colleges
Commercialisation i.e. actual revenues
Programme effectiveness & impact
Assessment of wider economic impacts
Programme Management & Monitoring
International Comparative
Position of SFI
Assessment of research outputs
& quality
Discussion of peer review process
User satisfaction with management and implementation based on an extensive survey – note survey response ratios
Discussion of approaches to monitoring
Programme effectiveness & impact
Assessment of wider economic impacts
Programme Management & Monitoring
International Comparative
Position of SFI
Assessment of research outputs
& quality
Based on discussions with a list of international academics
The concept of “virtual citations”
Conclusions
• The report concludes that a discontinuation of the programmes would adversely affect the buildup of research capacity
• It observes that SFI has unusually high staff turnover
• It points out room for improvement in interaction with other agencies, in particular the HEA
• It notes that 1.4 billion Euro spent so far and that VFM and financial sustainability will be of increasing importance
Recommendations
Issues
Some independent or disinterested input would
be useful. Maybe some comparisons with
international or European benchmarks for countries of similar size would be an idea. But sure that’d
be just mad, Ted.
There has been an increase in PhDs but there is no clarity about the counterfactual or control methods.
No return on investment can be expected in the short run. This is assessed by recipients of SFI funding only.
No serious international bench-marking. Only comparator is Irish programmes not funded by SFI. “Virtual impact” particularly worrying because of this.
Methodology merely repeats the objectives so not clear how objectives of review will be met or that they have been.
Recommended