Gemma Rauret

Director

Quality Assurance for Quality Assurance for Enhancement: a case study of Enhancement: a case study of

ANECA (Spain)ANECA (Spain)

INQAAHE Conference 2009INQAAHE Conference 2009

30st March – 2nd April 200930st March – 2nd April 2009

Abu Dhabi, United Arab EmiratesAbu Dhabi, United Arab Emirates

Tensions and challenges in the evaluation process

Simple vs. diversified models for assuring quality

The Spanish model for quality assurance

The new quality assurance approach

Table of contentsTable of contents

1. Tensions and challenges in the Tensions and challenges in the evaluation process in QAevaluation process in QA

1. Is it feasible the evaluation

enhancement-oriented model?

2. Is it possible to harmonize an

evaluation enhancement-oriented to

another accreditation-oriented?

3. How to integrate an evaluation for

accreditation with an evaluation for

enhancement?

Evaluation Process in QA: tensions and challenges

Accountability vs. Quality Enhancement Governmental Regulation vs. Self-regulation Performance Indicators vs. Expert Opinion Accreditation vs. On-going Quality Enhancement

Can both approaches be integrated?

Purist perspective vs. Practical position Stakeholders’ claims Accreditation legitimacy Integration in a supra-national framework : the

EHEA

Tensions in the two approaches in the evaluation

Evaluation Process in QA: tensions and challenges

• Higher Education Institutions: HEIs are responsible for the quality of their

educational offer HEIs must be able to show quality both at the

national and International level A quality culture within HEIs must be promoted Quality of academic programmes for students

must be developed and enhanced

Basic principles that rule the ESG

Evaluation Process in QA: tensions and challenges

• Society: Society’s interests concerning quality of

higher education must be assured and safeguarded

Process: Quality assurance accountability-oriented is compatible Quality assurance accountability-oriented is compatible

with with enhancementenhancement

Processes should be compatible with diversity and innovation

Transparency and external experts are significant within the quality assurance processes

Processes allowing HEIs can show their responsibility concerning accountability for both public and private funds received should be developed

Evaluation Process in QA: tensions and challenges

Scope and Goals of the Evaluation: National Higher Education Academic Rules Programme and/or institutional accreditation Users’ protection Public access to the information about programmes

and institutions Quality improvement and enhancement

2. Simple versus diversified models Simple versus diversified models

for assuring qualityfor assuring quality

Simple or different evaluation models? External and internal determining factors

Reality and evaluation culture of the system Complexity, volume, tradition and legal frameworks Problems to be undertaken Potential of the organization to accomplish the evaluation

– Internal quality units in the HEIs and

– quality assurance agencies involved in the process

Common aspects of both models An agency that co-ordinates the process A self-evaluation report A peer site-visit A public final report

Evaluation Process in QA: tensions and challenges

Stages where different problems will be undertaken/ the role played by the quality assurance/nature of the external review

Doubts regarding educational standards/ identification of programs placed under that threshold/ accreditation

Doubts regarding the efficiency of either the Higher Education system or the HEIs / Public accountability and creating awareness in institutions/ranking or report to state

Evaluation Process in QA: tensions and challenges

Doubts regarding Innovation capacity and QA capacity / stimulate self-regulation capacity of institutions. Public accountability/Audit to the institution

Need to stimulate sustainable Quality Culture in Institutions/ Improvement based on self regulation or public accountability /audit report to the institution

Decreasing Comparative Transparency across HE Systems/stakeholder information/ publication of performance indicators

Evaluation Process in QA: tensions and challenges

What kind of agency can provide responses to the challenges set up?

The nature of an agency : learning organization shared know-how critical and independent thought

The technology of the agency: Guides, standards and benchmarks

The process Interaction with the HEI: moment and type of

interaction Ensuring the presence of different stakeholders

Evaluation Process in QA: tensions and challenges

Evaluation Process in QA: tensions and challenges

What kind of profile, training and role place the experts and agency’s staff in evaluation?

Experts

– Highly qualified

– Trained on the evaluation model to be applied

– International experience Agency’s staff

– Highly qualified staff

– On-going training: networking with other agencies’ staff

– Network of collaborating experts

3. The Spanish model for quality The Spanish model for quality

assuranceassurance

Spanish Higher Education System

UniversitiesUniversities

7575 universities all over Spain universities all over Spain

(50 public – 25 private)(50 public – 25 private)

Public universities – 1.326.734

Private universities – 134.743

Total number of students enrolled 1.461.477

The Spanish SettingThe Spanish Setting

Programme evaluation for improvement was

introduced but without consequences (1985-2005).

The initial reaction was very stimulated although due to the lack of consequences the impact was rather unequal.

A stage without consequences is useful but cannot be extended for an indefinite period.

This stage led to the creation of QA agencies at the regional and the national level and QA units in HEIs.

The Spanish model for QA

The Spanish Setting (2)The Spanish Setting (2)

The EHEA and the quality assurance (2007 onwards).

Problem to be undertaken and evaluation model

Ensuring that all official programs fulfilled the standards established in the legal framework.

Adopting the accreditation model with a few elements enhance-oriented

Adopting a national evaluation model for the teaching staff

Mandatory for bachelor and master programs

Implies funding and recognition of degrees for the public service

ANECA and other regional agencies are responsible for the external evaluation

The Spanish model for QA

Design

Programme design

UNIVERSITY

The programme accreditation processThe programme accreditation process

Accreditation

Fulfilment evaluation

Council Univ.

Follow-up

Implementation

UNIVERSItY

ANECA and regional Agencies

Ex – ante Evaluation

Verification

Council Univ.

ANECA

The Spanish model for QA

• How ANECA has undertaken the new challenge? ANECA as a learning organisation

Innovation Area Involvement at the International level

ENQA, ECA, RIACES and INQAAHE Taking part in innovative projects

ANECA’s technology

What Tools? Guide for universities Guide for evaluation

Methodology, standards and benchmarks

How are they created? Innovation Unit Technical Committee for validation Approval by the Board of Directors

The Spanish model for QA

The methodology: An evaluation procedure in three stages

Provisional Report oriented towards the fulfilment of the standards and improvement

Response of universities and enhancement Final Report on the fulfilment of the

standards

ESG Fulfilment

External reviewed in 2007: ENQA full membership

Included in the European Register for Quality Assurance Agencies (EQAR): December 2008

The Spanish model for QA

Profile and training of the agents involved in the evaluation Experts profile

Academic peers: expertise in QA, in programme evaluation, in International teams and members of the university staff

Professional Experts in the discipline specific field Students coming from the discipline specific field at the

suggestion of student unions Experts in quality assurance

Training Specific training sessions in the agency On the university system of Spain On the running of the evaluation procedure

The Spanish model for QA

Profile, training and role of the technical staff Profile

Expert in quality assurance Young academic staff with experience in quality assurance of

higher education Training subjects

On the university system of Spain On the running of the evaluation procedure On quality assurance Systems and the application of standards Data processing

Role played by the technical staff of ANECA Secretary of the committee Improve the reference book to be used by the Experts in the

committees Co-ordinates the organisation of the meetings

The Spanish model for QA

4. The new quality assurance The new quality assurance

approachapproach

The main features:

– Strengthening the internal QA system of HEI

– A public information system– To give importance to

improvement plans– Evaluation agencies should

adopt a consultancy role– Funding as an incentive for on-going

improvement

The new QA approach

The internal QA Systems of the institution Internal systems of the HEI are the only element really

sustainable

Based on on-going improvement Towards international quality levels

Best practices for benchmarking

Reporting on progresses Improvement of teaching quality

Quality system core

Staff involvement at all levels Public information system

Characteristics of the programs Planning and implementation of the learning process Performance indicators

The new QA approach

Quality improvement plan

Characteristics

Focused on learning quality enhancement Built up from self assessment Relevant actions for achieving the

improvement goals Feasible: human and material resources Adequate time

Institutional commitment Accountability of results

The new QA approach

Conclusions remarks

The evaluation approach improvement- and voluntary-oriented creates COMMITMENT

The evaluation approach accountability- and mandatory-oriented only creates FULFILMENT

Conclusions

Both are necessary components for generating

a quality culture

www.aneca.eswww.aneca.es

Thanks for your Thanks for your attentionattention