ERA 2012

A funny thing happened on the way to certification

Leanne HarveyExecutive General Manager

ERA submission process• Universities upload their xml files into SEER• SEER checks files against verification and

business rules• Universities amend and re-submit• Once rules are passed, ERA staff conduct

human checks and audits of submissions• Submissions are certified by universities

→SEER then performs a range of analysis to present to Research Evaluation Committee members

How did Submission go?• For the majority of Universities the process went

smoothly• Improved SEER processing time meant universities had

time to upload and re-upload submissions as needed• Case management approach (individual ARC staff working

with individual research office staff) worked well• ‘Representative spread’ caused some confusion for some

universities, though may got it right first time. We will look at guidance provided for the next round

• Explanatory statements generally much improved, though some still focusing on PR puff rather than information.

• FoR coding of outputs sometimes has to be questioned – the ARC will police inappropriate assignment outputs

ERA 2012 – research outputs nominated for peer review available in repository

What did reviewers think?• Generally, repositories worked well• Some universities still had very confusing repository

setup – multiple clicks and “small target” links• Quality of scans sometimes an issue• (Still) need to consider the evidence for non-creative

works – there were still too many cases of flyers rather than evidence of research quality – it’s hard to get a good quality rating without evidence of quality

• Research statements also need thought – they are not PR pieces, need to draw out the research content

ERA 2012 outcomes

Release of ERA 2012 results• As of 2010, universities will have their own results under

embargo before the public release. This means you will receive• Your own ratings in a spreadsheet;• You will not receive other universities’ ratings under embargo,

nor the full national report for the same reason• After the embargo period, the full National Report will be

released• National Report will contain righ data analysis as well as ratings

for all units of evaluation at all universities• The ERA 2012 Evaluation Handbook and Peer Reviewer

Handbook will also be released to the public• Info sessions available on request – please contact the ARC if

you want staff to visit your university to give talks and answer questions about the ERA results or the process

Access to dashboard and data• Institutions will be given access to their evaluation

dashboards in SEER for a period of three months.• Access will be given via ERA Liaison Officers shortly after

the public release of results• The dashboards are not public information, as they

contain a range of proprietary data• Within each university, access is up to you to decide – but

the information cannot be shared outside of your university

• You will also be provided with equivalent data on spreadsheets – these will outlast the dashboard access, but similarly cannot be shared or made public

Interdisciplinary Research(as indicated by co-apportioned papers)

Authorship patterns– all output types

Research Outputs patterns – all output types

Research evaluation beyond ERA 2012

• Timing of the next ERA round has not yet been announced

• We are always looking to expand the indicator set where appropriate → discipline consultaations

ERA 2012 National Report

Questions?