News from the Granting Agencies / Nouvelles des

agences subventionnairesNSERC/CRSNG

CAGS / ACES51st Annual Meeting / 51e Congrès

annuelMontréal, November 3-6, 2013

Outline / Aperçu• Program Evaluations

– Canada Graduate Scholarships (tri-agency)– PGS/IPS/CREATE

• Outcome of consultation on the Postdoctoral Fellowships Program

• Évaluations de programme– Bourses d’études supérieures du Canada (tri-agence)– BES/BESII/FONCER

• Résultat de la consultation concernant le programme de bourses postdoctorales

2

Program Evaluations - Purpose

• To meet the requirements of the Federal Accountability Act and the Policy on Evaluation– Every program has to be evaluated every five years

• To provide insightful, actionable findings, about program performance and relevance

• To provide recommendations for future improvements

3

Évaluation des BESC - Échéancier

• Rapport d’esquisse approuvé par les comités consultatifs

• Les divisions d’évaluation des agences préparent les méthodes de collecte de données

• Date prévue pour compléter l’évaluation: mai 2014

• L’évaluation couvrira les années fiscales 2008-2009 à 2012-13– Considération des résultats de l’évaluation

précédente– Questions clés développées en consultation avec le

personnel des agences et la collectivité– Méthodes d’évaluation multiples afin d’assurer des

résultats crédibles et solides

5

CGS Evaluation - Objectives

• Assess the outcomes of CGS recipients in comparison to non-recipients

• Assess the longer term impact of the CGS program through a trajectory analysis– The extent of the program’s contributions to a

required supply of HQP• Identify « unexpected » outcomes to help

determine the program’s suitability – Inform decisions about what the « expected »

outcomes should be– Compare data to identify any differences across the

three agencies6

Évaluation BESC – Objectifs (con’t)

• Déterminer si des changements ont eu lieu dans l’expérience des récipiendaires depuis la dernière évaluation

• Apprendre des différents mécanismes de livraison du programme des agences subventionnaires afin d’informer le processus d’harmonization

• Coordonner la collection de données avec les autres évaluations présentement en cours– Afin de diminuer la charge des répondants

8

CGS Evaluation – Key Questions

• To what extent is the program achieving its expected immediate outcomes?– Incentives for enrolment in graduate studies– Incentives for students to complete studies in a

timely fashion• To what extent is the program achieving its

expected intermediate outcomes?– Increased numbers of students completing studies– High quality research training; attract and retain

• What are the outcomes achieved by recipients?– Research output, career progression, etc.

9

CGS Evaluation – Key Questions (con’t)

• Is the program delivered in a cost-effective manner?– Are there any best practices that could inform the

harmonization project?• Does the program remain relevant? Is there a

continued need?• Does the program align with roles,

responsibilities and priorities of government?

10

CGS Evaluation – Discussion/Feedback

• How has the graduate education landscape evolved in the last five years?

• Are there emerging issues in graduate education in Canada that have bearing on the current evaluation?

• Comment l’environnement pertinent aux études supérieurs a-t-il évolué dans les cinq dernières années?

• Quels sont les enjeux qui pourraient être pertinents pour l’évaluation en cours?

11

CREATE/PGS/IPS Evaluation

• At a much earlier stage than the CGS evaluation

• Evaluation will be combined and expected completion is for spring 2015

• Evaluation objectives are similar to that of the CGS evaluation– Especially for PGS and IPS

• At the stage of determining what the evaluation questions will be– A stronger focus on CREATE

• As with CGS, a mix of quantitative and qualitative data gathering methods will be used 12

CREATE/PGS/IPS – Questions

• Draft questions are similar to the CGS evaluation questions

• The CREATE program does trigger different questions– To what extent are high caliber HQP participating in

the CREATE, PGS and IPS programs?– To what extent does CREATE provide a viable option

for providing both research and professional skills training?

– To what extent does CREATE complement existing university initiatives?

13

Postdoctoral Fellowships Issues

• Following a positive evaluation of the PDF program, recommendations and focus on three issues– Value of the fellowship (currently at $40,000)– Timing: application deadline and announcement of

results– Application pressure

• Input was requested from stakeholders– CAGS and CAGS Board of Directors (C-TACC)– CAPS and CAPA– Engagement and Regional Visits

• Upcoming discussion at NSERC’s Committee on Grants and Scholarships (COGS) 14

PDF – Date limite et annonce des résultats

• Le CRSNG s’est engagé à annoncer les résultats du concours des postdoc plus tôt

• La plupart des répondants préfèrent une date limite du 1er octobre– Plus tôt qu’octobre comporte des difficultés liées au

début du semestre d’autome• Le CRSNG devra être plus efficace avec le

traitement des demandes ainsi qu’avec leur transmission aux comités de sélection

16

PDF – Value of the Fellowship

• PDF value is currently set at $40,000/yr• NSERC was considering asking the university

or supervisor to top up the award• Majority opinion was that a top-up should not

be mandatory– Average discovery grant not being high enough– Limiting financial capabilities of researchers and

institutions– A top-up would undermine the purpose of the

program, a prestigious award that offers independent training

• NSERC to work towards increasing the value?17

PDF – Application Pressure

• NSERC reduced, from 2 to 1, the number of applications an individual could submit

• Rationale was to ensure a more efficient use of time for applicants and reviewers

• Following unfavourable feed-back, NSERC agreed to re-consider its decision

• There was no emerging trend in feedback received– Providing quotas to universities– Pre-screening of applications– Allow highly rated unsuccessful applicants to re-

apply18

Questions/CommentsQuestions/commentaires

19