67
Primiano Di Nauta Pirjo-Liisa Omar Angelika Schade J. P. Scheele (eds.) Accreditation Models in Higher Education Experiences and Perspectives ENQA Workshop Reports 3 European Network for Quality Assurance in Higher Education Helsinki

Accreditation Models in Higher Education · Accreditation Models in Higher Education – Ex-periences and Perspectives in Rome, Italy. ... 2.1 AUSTRIA: Accreditation in Austrian Fachhochschule

Embed Size (px)

Citation preview

Primiano Di NautaPirjo-Liisa OmarAngelika Schade

J. P. Scheele(eds.)

Accreditation Models in Higher Education

Experiences and Perspectives

ENQA Workshop Reports 3

European Network for Quality Assurance in Higher Education • Helsinki

The views expressed in this Workshop Report are those of the authors.

© European Network for Quality Assurance in Higher Education 2004, Helsinki

This publication may be photocopied or otherwise reproduced without permission of the publisher.

Cover: Jussi Hirvi / Green Spot Media FarmLayout: Anja Kauppi / Pikseri Julkaisupalvelut

ISBN (paperbound): 952-5539-00-8 ISBN (pdf): 952-5539-01-6 ISSN 1458-106X

Printed by MultiprintHelsinki, Finland, 2004

Foreword

In November 2003, ENQA organised in co-operation with its Italian member agency, Comitato per la

Valutazione del Sistema Universitario, a workshop on Accreditation Models in Higher Education – Ex-

periences and Perspectives in Rome, Italy. The intention of the workshop was to understand the princi-

ples of the various accreditation methodologies and models in higher education, to demonstrate examples

of possible good practices on the basis of country case studies, to discuss and evaluate the usefulness of

accreditation activities, their strengths and weaknesses, and to make recommendations for the future.

Accreditation is a complex concept and wide-ranging as a practice, without one single denotation. In

fact, national adaptations and practices vary considerably. Exactly for this reason, the workshop proved

to be a successful forum for learning from each other in terms of good accreditation practices and expe-

riences. In addition, the dialogue on the nature, purpose and objectives of accreditation, and an examina-

tion of the weak and strong points contributed to a greater understanding of accreditation in a European

context. Moreover, the workshop considered the differences between accreditation and evaluation, find-

ing that both of these modes of operation aiming to assure quality in higher education contain a develop-

mental aspect in them.

The Rome workshop contributions have been collected to this workshop report. Unfortunately, be-

cause of limited space, the individual contributions by the workshop guest speakers have had to been

shortened. However, the full version of each of the workshop presentations, including the PowerPoint

presentations that are not included here, can be found on the ENQA website (http://www.enqa.net/

workshop_rome.lasso). The report is structured so that it begins with three general articles on accredita-

tion. This is followed by seven country cases and finally, a conclusion on the contribution of the work-

shop.

It is my hope that this report will be of use to many, enhancing further the benefits of the workshop by

continuing the sharing of ideas and fruitful discussion on accreditation, its methodologies and practices.

Through the publication of this report ENQA wishes to emphasise its support for accreditation as an im-

portant method of the overall concept of quality assurance.

Christian Thune

President

ENQA Board

Contents

1 On Accreditation ___________________________________________________________5

1.1 The Power of Accreditation: Views of Academics (Lee Harvey) ____________________5

1.2 Licence to Kill: About Accreditation Issues and James Bond (Ko Scheele) __________19

1.3 Premises to Accreditation: A Minimum Set of Accreditation Requirements

(Muzio M. Gola) _____________________________________________________________25

2 Country Cases ____________________________________________________________32

2.1 AUSTRIA: Accreditation in Austrian Fachhochschule Sector (Kurt Sohm) ___________32

2.2 FINLAND: Accreditation Models in Higher Education in Finland: Experiences and

Perspectives (Kirsi Mustonen & Sirpa Moitus) ___________________________________35

2.3 FRANCE: Evaluation and Accreditation Practices in France. A Powerful Tool

for Improving Quality in Engineering Education (René-Paul Martin) ________________39

2.4 GERMANY: Quality Assurance and Accreditation in Germany

(Hans-Uwe Erichsen) ________________________________________________________42

2.5 HUNGARY: Accreditation Models in Higher Education: Experiences

and Perspectives (Christina Rozsnyai) _________________________________________46

2.6 ITALY: From Authorization to Accreditation – A Difficult Path for the University

System in a Changing Society (Alessandro Figà Talamanca)______________________48

2.7 An Approach to Accreditation: The Path of the Italian Higher Education

(Carlo Calandra Buonaura & Primiano Di Nauta) _________________________________51

2.8 THE NETHERLANDS: Accreditation in the Netherlands (Karl Dittrich) ______________55

2.9 NORWAY: Quality Assurance of Norwegian Higher Education

(Tove Blytt Holmen)__________________________________________________________59

3 Conclusion ________________________________________________________________62

Appendix________________________________________________________________________65

ENQA Workshop Reports

1 On Accreditation

1.1 The Power of Accreditation: Views of Academics

Professor Lee Harvey, Centre for Research and Evaluation, Sheffield Hallam University, [email protected]

Introduction

The paper draws on many years’ experience ofanalysing external evaluations of quality andstandards. It is important to note that quality andstandards are not the same (Harvey and Knight,1996). The paper will draw on the views of thosewho have been involved in accreditation in Britainand in North America. Both countries have hadforms of accreditation for decades. These viewswill, at a surface level, help to identify the per-ceived benefits and problems of accreditation.However, those same views, when criticallydeconstructed, will also raise fundamental issuesabout accreditation.

Overall, the view underpinning this paper is thatEurope is rushing precipitously into accreditationand that the approach being taken is based on naïveviews of what accreditation is and what it canachieve. More fundamentally, there is an underly-ing but unspecified and unexamined set of taken-for-granteds that legitimate accreditation. Accred-itation is neither neutral nor benign; it is not apo-litical. Quite the contrary, the accreditation route ishighly political and is fundamentally about a shiftof power but a shift concealed behind a new publicmanagement ideology cloaked in consumerist de-mand and European conformity. The paper intendsto demonstrate this.

Also, accreditation is not a process somehow setaside from audit, assessment, or standards moni-toring such as external examining. Accreditation

uses methods and has purposes that overlap withaudit, assessment and external examining.

Accreditation

Before exploring these issues, some orientation onwhat accreditation is and how it relates to otherexternal processes. Accreditation may be of pro-grammes or institutions. Accreditation is theestablishment or re-statement of the status, legiti-macy or appropriateness of an institution, pro-gramme (i.e. composite of modules) or module ofstudy.

Institutional accreditation

Institutional accreditation effectively provides alicence to operate. It is usually based on an evalu-ation of whether the institution meets specifiedminimum (input) standards such as staff qualifica-tions, research activities, student intake and learn-ing resources. It might also be based on an estima-tion of the potential for the institution to producegraduates that meet explicit or implicit academicstandard or professional competence.

Institutional accreditation or re-accreditation, inEurope for example, is usually undertaken by na-tional bodies either government departments orgovernment-initiated agencies that make formaljudgements on recognition. In some countries,with a total or preponderant public sector highereducation system, there is little need for institu-tional accreditation, per se, but there is a growingneed for a mechanism to validate ‘upgrading’ ofnon-university higher education institution (suchas colleges, polytechnics, Fachhochschule) to uni-versity status, as has happened, for example, in theUK and Sweden.

In the United States, with a large private sector,accreditation is a self-regulatory process of recog-nition of institutional viability by non-governmen-tal voluntary associations.

5

ENQA Workshop Reports

The focus of US institutional accreditation haschanged. Initially accreditation was a device ‘usedby a college or university to convince other institu-tions that its students and courses should be ac-cepted by them, and vice versa’ (Murray, 2002,p.1). It evolved into a form of public accountabilityproviding assurances ‘to those outside the highereducation community as well as those inside it thatthe institution had capacity to offer its programs’(Murray 2002, p.1). Furthermore, despite the vol-untary nature of the process, there has been a fund-ing link through eligibility for federal aid. Increas-ingly, there has been a shift in accreditation to fo-cus more on outputs, in particular, student learningoutcomes.

In Canada, the government of Ontario has estab-lished The Post-secondary Education Quality As-sessment Board to examine applications to offerdegrees from institutions other than the provinces’publicly-funded universities (INQAAHE, 2001a).Institutional accreditation, especially initial recog-nition, tends to be more prominent in countrieswith a significant new private higher educationprovision, such as those in the Americas and East-ern Europe.

Delegates at the international seminar The Endof Quality? accepted that institutional accredita-tion was useful, in theory, to ensure the integrity ofhigher education – including international integri-ty. However, the context and stage of developmentof higher education within any system is a key var-iable in determining the importance of accredita-tion. The more new (private) development themore, it was thought, is the need for institutionalaccreditation.

Programme accreditation

Programmes may be accredited for their academicstanding or they may be accredited to produce grad-uates with professional competence to practice;usually referred to as professional accreditation.

Accreditation (and re-accreditation) of coursesin North America tends to focus on professionalareas. About 14 different non-governmental vol-untary associations recognise provision in institu-tions that have been found to meet stated criteria of

quality. These accreditors judge whether the studyprogrammes appropriately prepare graduates toenter a profession. Accreditation of programmes inthe USA is, thus, linked to providing a licence topractice but is separate from it.

This is very similar to the role played by the pro-fessional and regulatory bodies in the UK, whoalso control access to the profession by making ac-creditation of the programme a prerequisite forgraduate entry. Perhaps more draconian than theirUS counterparts, some bodies in the UK set andgrade their own examinations and require a periodof work experience before registering graduates asfull professionals (Harvey & Mason, 1995).

The newer accreditation in Eastern Europeancountries such as Hungary, the Czech Republicand Slovakia has, at least initially, opted for pro-gramme accreditation in all academic fields (West-erheijden, 2001). This appears to be designed prin-cipally to provide academic rather than profession-al accreditation in the wake of the Soviet era. Themushrooming of new programme accreditationproposals in some Western European countries,linked to bachelor-masters conversion, also pre-dominantly appears to be academic accreditation.Instead of accrediting institutional processes forthe creation of bachelor-masters courses based onexisting provision, it seems some countries intendto accredit every new programme.

Licence to practice and accreditation

There is a distinction between graduating from anaccredited programme and having a licence topractice. In some cases, these are coincident, espe-cially for graduates from some postgraduate pro-grammes. Sometimes an undergraduate degree in aspecified subject is a prerequisite for progressionto a postgraduate course or diploma in that area. Insome cases any good undergraduate degree is apre-requisite for further professional training; forexample, in law in the UK there is a one-year post-graduate conversion course that non-law graduatestake before joining the law society postgraduatequalification programme.

In many professional areas, graduation from anappropriately accredited academic programme is a

6

ENQA Workshop Reports

preliminary step and full professional certification,and thus a licence to practice, follows only aftersome period of work experience. In some instanc-es, such as teaching, a licence to practice may bevirtually independent of studying on an accreditedprogramme. In many US states, obtaining a teach-ing licence is not dependent on having an accredit-ed teacher education degree.

Validation and accreditation

When examining subject or programme accredita-tion it is important also to distinguish between val-idation, revalidation, accreditation and re-accredi-tation. Validation refers to internal processes ininstitutions. So, a validation process would ensurethat a new programme fulfilled internal institu-tional criteria. Validation is the internal acknowl-edgement of the establishment and legitimacy of aprogramme. In some countries, such as the UK, theintroduction of new programmes of study and newcomponent modules in some areas, such as socialscience, is solely an internal process. In others,new programmes require external approval, froman agency or government department and if theyare in ‘professional’ areas they may need addi-tional accreditation. In other countries, there arelimits on new developments, for example, in Nor-way, if a subject area is already well-established atan institution, new programmes up to 90 credits(1.5 years) can be opened.

Revalidation is the formal renewal of that ac-knowledgement. Most institutions have processesfor periodic review of existing programmes ofstudy and of their constituent modules. This proc-ess may be linked to external accreditation but isoften an internal process within permitted parame-ters and, usually, conforming to explicit guide-lines. External re-accreditation may be ‘delegated’to the internal revalidation procedure (usually onthe condition that the internal procedure uses ex-ternal advisors).

Accreditation is the formal or official externalrecognition of a (validated) programme. This maybe for funding purposes or it may be registration ofthe programme as a provider of professional edu-cation (which thereby signifies that graduates have

attained a level of minimum professional compe-tence). The external accreditation agency may be anational agency or a discipline-specific agency ora regulatory or professional body with delegatedauthority. Re-accreditation is, thus, the formal re-newal of an accredited programme.

Accreditation criteria and decisions

Accreditation has been described as a public state-ment that a certain threshold of quality has beenachieved or surpassed (Campbell et al., 2000; Krist-offersen, Sursock, & Westerheijden, 1998). How-ever, one might argue that accreditation is moreabout minimum standards (be they academic, com-petence, service or organisational (Harvey, 1999))than about the quality of the process. None the less,accreditation decisions are, or at least should be,based on transparent agreed, pre-defined standardsor criteria (El-Khawas, 1998; Sursock, 2000).

Accreditation is a binary state, either a pro-gramme or an institution is accredited, or it is not(Haakstad, 2001, p. 77). However, the absolute ofthis binary state is blurred or softened by a ‘hold-ing’ decision that permits, in effect, progression toaccreditation. This ranges from accreditation sub-ject to further action, through probationary accred-itation to permission to reapply for accreditation.

Focus of accreditation

Accreditation may be focused on inputs, process oroutputs or any combination of these.

Programme accreditation tends to focus on in-puts such as staffing, programme resources, andcurricula design and content. Sometimes it ad-dresses the teaching process and the level of stu-dent support. Occasionally programme accredita-tion explores outcomes such as the graduate abili-ties and employability. In some cases, the mediumof delivery might be the key focus, especiallywhen it differs from the norm.

Institutional accreditation tends to focus on theoverall infrastructure, especially the physicalspace, along with the IT and library resources andthe staffing. It might address this from the point ofview of the overall student learning experience. Inaddition, institutional accreditation might focus on

7

ENQA Workshop Reports

financial arrangements and viability, governanceand regulation and administrative support. Wherean institution offers distance or on-line learning,the medium of delivery might be a focus of accred-itation procedures. Increasingly, the US regionalinstitutional accreditation agencies are focusing onoutcomes and effectiveness.

In principal, though, rather than the input-proc-ess-output focus, accreditation might be based onrecognition that the institution has in place appro-priate control and monitoring processes to ensuresatisfactory quality and standards. However, iden-tifying appropriate mechanisms is normallyviewed as an auditing function distinct from, butpossibly contributing to, a formal process of ac-creditation of an institution. However, the term(quality) audit is not restricted to an exploration oforganisational process.

Rationale

Rather more complex is the ostensive rationale foraccreditation. Accreditation is primarily aboutcontrol of the sector; this is much more explicit inaccreditation than in other external quality proc-esses such as audit, assessment or external exam-ining. Although accreditation involves complianceand indirect accountability, its main function is tomaintain control of the sector and the programmesoffered. Improvement is a spin-off from accredita-tion processes, which some agencies emphasisemore than others. Institutional accreditation isdesigned to ensure that institutions of dubiousmerit do not become established as bona fidehigher education institutions. Accreditation alsomonitors the sector to ensure that accredited insti-tutions continue to fulfil the expectations of a uni-versity or college. A key concern is the need tocontrol ‘for-profit’ organisations, whose motiva-tion is different from the public sector.

In many countries, with a predominant publicsector higher education system, there is little or noinstitutional accreditation per se, but there hasbeen a growing tendency, fuelled by new publicmanagement ideology, to require institutions todemonstrate accountability for public funds. Al-though not the same as accreditation, in the ex-

treme, failure to exhibit satisfactory accountabilitycan result in the ‘de-accreditation’ in the form ofclosure or merger of unsatisfactory institutions, ashas happened in the further education college sec-tor in the UK.

Accreditation at the programme level is alsoabout control. In Eastern Europe, academic ac-creditation of programmes is about ensuring ade-quate standards, a function fulfilled, in effect, inthe UK (and some other Commonwealth coun-tries) by the external examining system. Althoughthe latter is not accreditation per se, unsatisfactoryexaminers reports might lead to the closure orsanctioning of a programme either by the institu-tion management or as a result of other forms ofexternal monitoring such as external subject re-view or academic audit.

Professional accreditation is even more aboutostensive control. It is about an external agencymaintaining control of a subject area that links intoprofessional employment, especially where topractice requires certification separate from aca-demic qualification. Although such bodies provideguidelines with which successful accreditees com-ply, these guidelines are manifestations of the or-ganisation’s control of the sector. Sometimes thiscontrol is grounded in legislation, such as the Brit-ish General Medical Council’s regulatory func-tion. Sometimes, despite having no regulatorypower, the professional body is so well establishedin the profession that it is impossible to gain workin some areas without it.

Accreditation methods and mechanisms

Accreditation involves a set of proceduresdesigned to gather evidence to enable a decision tobe made about whether the institution or pro-gramme should be granted accredited status. Theonus is on the applicants to ‘prove’ their suitabil-ity; that they fulfil minimum criteria. Methods bywhich this evidence is gathered overlap with meth-ods used in audits, assessments and external exam-ining. The component methods include self-assessments, document analysis, scrutiny of per-formance indicators, peer visits, inspections, spe-cially-constituted panels, delegated responsibility

8

ENQA Workshop Reports

to internal panels often via proxy entrustment toexternal examiners or advisors; stakeholder sur-veys, such as student satisfaction surveys, alumniand employer surveys, direct intervention, such asdirect observation of classroom teaching or grad-ing of student work.

Accreditation, audit, assessment and external examining

Although accreditation is distinct from audit,assessment and external examining there is adegree of overlap between these different externalprocesses (Harvey, 2002b; Stensaker, 2003).

One big difference, though, is that audit, assess-ment and external examining operate on thepremise that the institution or programme is func-tioning appropriately and the external process hasto demonstrate otherwise (innocent until provedguilty model). For example, audits often involve amethodology designed to test the verisimilitude ofinstitutional or programme claims. Accreditation,though, shifts this round and institutions or pro-grammes have to prove that they are worthy (guiltyuntil proved innocent model).

Nuances

Accreditation thus has three nuances. First, accred-itation as a process applied to applicant organisa-tions. Second, accreditation is the label that institu-tions or programmes may acquire as a result of theaccreditation procedures. Third, underpinning thefirst two, accreditation is an ‘abstract notion of aformal authorising power’ (Haakstad, 2001, p. 77),enacted via official decisions about recognition(the accreditation process). It is this underpinningabstraction that gives accreditation its legitimacy.Ironically, this abstraction, frequently taken-for-granted, is not a traditionally intrinsic aspect ofaccreditation. As Jones (2002, p.1) has pointed out,‘The original audience for accreditation was theacademy itself. The process did not arise inresponse to concerns about quality expressed byexternal audiences…’

This third nuance chimes with the issue, alludedto throughout the forgoing, of the underpinning

ideology and politics of accreditation. It leads to aninvestigation of the power relationships embodiedin the accreditation process. An examination of theperceptions of those who have engaged with ac-creditation of various types reveals surface viewsabout the benefits and draw-backs. A second-orderexamination of the comments will, though, alsouncover the political and ideological dimensions.

Professional and regulatory bodies (PRBs) playthree roles (Harvey & Mason, 1995). First, they areset up to safeguard the public interest. This is whatgives them their legitimacy. However, profession-al bodies also represent the interest of the profes-sional practitioners and here they act as a profes-sional association or trade union (including legiti-mating restrictive practices), or as a learned socie-ty contributing to continuous professional devel-opment.

Third, the professional or regulatory body repre-sents its own self-interest: the organisations act tomaintain their own privileged and powerful posi-tion as a controlling body. This is where control,legitimated by public interest becomes confoundedby control based on self-interest.

Views of participants

The following views are derived from what littleliterature there is on participants’ views of accred-itation and the responses of 53 academics andadministrators who have been involved in accredi-tation processes. This group, are mainly from theUK with some US, Canadian and Australian input.The qualitative perceptions were gathered on-linevia e-mail correspondence (including follow-updiscussions to clarify specific areas). The majorityof respondents comment on subject accreditationrather than institutional processes and their com-ments relate to 24 different discipline areas, as wellas accreditation of learning and teaching practicesper se. Quotes are included but for reasons of con-fidentiality the source is assigned a number, andthe remainder of the reference relates to countryand the subject area the respondent is talkingabout. As far as possible, the quote is contextual-ised without making it too long and AT TIMES

9

ENQA Workshop Reports

ostensibly deconstructed, using Barthesian 1 semi-ological notions of denotation, connotation and‘myth’, the latter informed by prevailing ideology.

Institutional accreditation is more of an issue inthe US than the UK. Programme accreditation inthe UK is enormously varied and about 100 regu-latory and professional bodies are involved insome form of accreditation of higher educationprogrammes in British higher education institu-tions (Harvey & Mason, 1995).

Necessity: employment and marketing

Many respondents were of the view that profes-sional accreditation was either necessary for pro-fessional employment, or enhanced the job pros-pects of, their graduates.

However, for most respondents this necessitywas closely linked to a concern that the marketa-bility of programmes in some areas is closely tiedto accredited status and that failure to achieve ac-creditation would be problematic:

For some respondents, accreditation was not justnecessary or a marketing device to get more studentsbut something that attracted better students. The al-lusion to real-world relevance is also recognised.

However, comments about real-world rele-vance, suggests that the accreditors are in tune withthe real world, which, as will be discussed below,is a moot point. One US respondent, though asked,‘were accreditation not tied to federal funding orprofessional licensure, would your institution orprogram seek it anyway?’ (R33, US, general)

Another US respondent seems to suggest thatperhaps they would:

I think the self-study provides an opportunity forthe institution to conduct a formative evaluation

and identify both strengths and areas for improve-ment. The accrediting team can offer a more sum-mative evaluation and an objective external per-spective that can potentially strengthen the institu-tion. (R36, US, general)

The assumption, here is that there is an objectiveexternal view that is the province of the externalaccrediting body. The ‘objectivity’, though, maybe tempered by the controlling function of theorganisation, itself possibly a function of its ownself-interest, as noted above.

Uniformity

A significant and often repeated rationale foraccreditation in some areas is uniformity acrossthe sector. The presumption is that uniformity isimportant and desirable and thus that all coursesshould ‘cover’ the same content. This assumes thatcovering the same course content equates with uni-formity of learning and understanding of the sub-ject area. The question remains, though whetherthe demand for uniformity is the professional bodysafeguarding the public, representing its members’interests or reinforcing its own status?

The assumption is that there is an external guid-ing hand that knows what’s best and that academiahas to conform to it. This ‘measuring up’ is notviewed negatively in this case, indeed it is seen asan opportunity for reflection.

An alternative view is less benign.

Sometimes it seems to be about how powerful theagencies are – the professional body or the institu-tion and I’ve had experience of it going bothways…. In relation to psychology, it initiallyresulted in inflexibility in relation to residentialschools – mandatory to get a named degree andthis disadvantaged women with childcare needs.We then renegotiated after much feedback andbecause student voted with their feet (didn’t signup) and we then found money to provide an alter-native, and an on-line experience was developed.(R8, UK, psychology)

Do we read this as safeguarding of the public or isthis inflexibility born of the society invoking its

1. Roland Barthes (1957), and subsequently other commen-tators such as Stuart Hall (1973) have argued that symbols(signs, words through to gesture and fashion) have a sur-face, first-order, denotation. Rose is a word denoting akind of flower. But also symbols connote something else(a rose may connote passion). The second-order, connota-tion is contextual, sometimes a rose might connote theBritish Labour Party. But underlying each connotation is athird-order myth. A rose connoting the British Labour par-ty represents the myth of benign nationalism. Others havetaken Barthes further and more pointedly linked the‘myth’ to prevailing ideological frameworks.

10

ENQA Workshop Reports

public security mission to reinforce its politicalpower and omniscience?

What would it matter if undergraduate psychol-ogy students on different degree courses took dif-ferent syllabuses taught in different ways? TonyGale (2002), ex- Honorary General Secretary ofthe British Psychology Society (BPS), argues that,given a first degree in psychology does not giveyou a licence to practice, the society accredits un-dergraduate courses for political reasons, whichhave little to do with public security or pedagogy.

Academic or practitioner

This leads to the relative influence of academicsand practitioners in each other’s realms. ‘There isoften a clear tension between academic prioritiesand professional ones in say engineering or socialwork’ (R30, general, UK). I think they [accredita-tion processes] are valuable when:

• they focus on the professional rather than the ac-ademic side of the programme (though it has tobe acknowledged that the boundary is usuallyfuzzy);

• they explicitly acknowledge that the studentsare being educated and not just trained for a pro-fession;

• they are conducted by peers (i.e. have at leastone academic on the panel alongside the practi-tioners);

• they ask to see only strictly essential documen-tation;

• they are willing to respect and take on trust theexpertise and judgements of, for example, exter-nal examiners.

They can be harmful and irritating, though, whenthe opposite of any of the above happens. I think itis a matter of particular concern when profes-sional bodies try to overrule academic judgementson academic matters, for example, curriculumdesign and content and assessment of academicaspects of the course. (R35, UK, speech and lan-guage pathology, pharmacy, engineering)

The principle concern here seems to be the per-ceived infringement of practitioners into the aca-

demic realm, notably requiring specific coursecontent, making demands about teaching andlearning approaches, as was noted with psychol-ogy, and even questioning assessment judgments.

The tail wagging dog analogy, used by Gale(2002) above, recurs in comments of respondents:

We have found the RICS accreditation an ongoingproblematic saga with [problems] because ofallegedly inadequate A-level points at entry. Wehave a large number of mature students for whomthis is irrelevant and besides pride ourselves inhow far we raise our students’ capacity during thedegree not how well qualified they are before theycome here. RICS, like so many other institutions,seem to be allowing the bureaucratic tail to waythe dog. The actual membership (dealers and auc-tioneers) claim to be horrified at the bureaucraticrequirements but seem powerless to [control] theireducational department. (R27, UK, fine arts valu-ation)

We had a particular problem with one Engineer-ing institution over regulations that allow Honoursdegrees to be awarded to students who have beenreassessed. I think we got into a tail wagging dogsituation and included a ‘notwithstanding’ clausein the regulations to appease the accrediting body.(R16, UK engineering)

I am also aware, from my old NVQ days of trainingand development, that meeting awarding bodyrequirements can galvanise centres into improvingnumerous aspects of their programmes, but canalso end up with the tail wagging the dog. (R3,UK, general)

It is curious that these respondents should all useexactly the same phrase about accreditation imply-ing a clear perception in the sector about confusionof the locus of decision making. It is also interest-ing that the analogy is used rather than any directstatement made about where the power should lie.Indeed, only one respondent, an administrator,actually directly talked about accrediting bodiesand academics struggling for power. It is almost asthough it is a taboo subject. Even the differentpolitical agendas embedded in the accreditation

11

ENQA Workshop Reports

process rarely seem to get publicly aired. The fol-lowing is unusual in being blunt about the politics:

I have been involved in visiting boards myself onbehalf of the RIBA and Architects RegistrationBoard and actually chaired one to [another uni-versity]. When I was chairman I suddenly becameaware of how many hidden agendas were in exist-ence and as we rejected the views of both the RIBAEducation chairman and their full-time officer asto what we should say about the school (before thevisit took place), I was not asked to chair a visitagain. (R9, UK, architecture)

The tension occurs mainly in three areas; pro-gramme content, programme delivery and bureau-cratic requirements. The issues around delivery areparticularly about contested control and conse-quent inhibition of innovation. Bureaucracy is asmuch about synchronicity of processes as it is bur-densome workloads and unnecessary require-ments.

Content

The issue over programme content is not so muchthe specification of what subjects should be taughtbut how restrictive that specification is perceived.Academics tend to think that externals/practition-ers only need specify what is an essential core thatwould enable a student to become a practitionerand then leave it to the academy to develop acoherent educational programme delivered in amanner that they consider is pedagogically sound.

Delivery and innovation

Some respondents thought there was potential inaccreditation to rethink and develop innovativeideas. However, despite the potential there was noguarantee, for various reasons, that this would hap-pen. The structural constraints and the member-ship of visiting panels impacted on innovation.

Overall it really depends on the accreditation bodyand level of the subject area. I found from my expe-rience that it was restrictive in some ways but therewere also opportunities to be innovative and coverthe same areas in a different way. (R4 UKaccounting)

Professional body accreditation does stimulateprogramme teams to rethink what they are doingand encourage innovation but often within a pre-defined area. For example, CIPD requirementshave stimulated higher education providers tobuild in competence outcomes to programmeswhich might otherwise be purely academic. (R1,UK, personnel and staff development)

Depends entirely on the organisation and indeedthe panel who visits. My experience has been RIBA[Royal Institute of British Architects] good anddevelopmental, as are RTPI [Royal Town Plan-ning Institute]. The Institute of EnvironmentalHealth – a real pain – they are policing things notdeveloping. The opportunity [to rethink pro-grammes] again depends – especially on thenature of the guidelines and degrees of discretion.Environmental Health is compliance; the othersmuch more developmental (R7, UK, architectureand environmental health)

How valuable it is must surely depend on how it ishandled, but my fear is that it can deteriorate toticking boxes and compliance. A notable exceptionis the accreditation of academics by ILTHE, whichis based on the SEDA scheme. This is wholly qual-itative. Hence it most emphatically can and doeslead to rethinking….The SEDA scheme has led toa substantial amount of innovation. (R39, UK,education)

This perception of the positive, innovatory effectsof the ILTHE process resonates with earlier viewsand is further endorsed by another respondent:

All the team thought that the discussions werereally useful and productive, and that the ILTHE[accreditation] processes were very worthwhile. Ialso think that the processes actually encourageinnovation, while at the same time helping innova-tors to think hard enough about how their ideasare actually going to work in practice. (R15, UK,education)

The key to the satisfaction with this process mightbe that it encourages self-reflection and drivesinnovative thinking, requiring academic legitima-tion rather than compliance. The control element,

12

ENQA Workshop Reports

in effect, is minimised and the trust, innovative andreflective element is maximised.

One respondent, referring to engineering, noted:

Constraints of accreditation seem to be on contentrather than style of delivery, so doesn’t necessarilyrestrict innovation in learning and teaching meth-ods. (R16, UK, engineering)

However, not all those involved with engineeringaccreditation agreed:

Let me be frank. I believe accreditation to be adead hand discouraging innovation and restrict-ing students in what they do. I would far prefer towork in the non-accredited courses (BSc or what-ever) then in the accredited ones (BEng or what-ever) because they can be so much more exciting.Sorry, but that is my considered opinion after 13years as a professor of engineering in the ‘formerpoly’ sector. (R19, UK, engineering)

I have found that the alleged demands of regula-tory/professional bodies does restrict the optionsfor the curriculum – for example, in Engineering,the B.Eng has restricted the University’s policy onfree choice modules (such as languages) asdemanding core modules for all modules as that isthe requirement of the professional body. This is adebatable point of course – it depends on what wassaid to the professional body by the engineeringstaff and as some of them are on the professionalbody anyway..... (R40, UK, engineering)

This sceptical response is consistent with a discus-sion below about the alliance some academicsmake with the professional bodies against theirinstitutions.

It was not just in engineering that respondentsthought accreditation stifled innovation. Respond-ents were particularly concerned about situationswhere accreditors went beyond content and maderequirements about delivery:

The Geological Society had just taken upon itself anew role as watchdog over professional qualifica-tions for geologists, and [my university] was in thevanguard, applying for accreditation of itscourses. Not all of the courses could actually be

accredited because the Geological Society putsome very stringent requirements on the fieldworkcomponent of an accreditable course…. My per-ception is that we believed that we had to do it toretain credibility and that it was indeed just a hoopto jump through. We even see accreditation as aforce for stasis, because it prevents us fromaccrediting innovative new courses that we mightwant to run (problems with rigid fieldworkrequirements, etc.). (R43, UK, geology)

The terminology here is instructive: ‘watchdog’and ‘hoop to jump through’ imply not only thecompliance requirement of the latter but also thatthe organisation set itself up as a controller of thediscipline, although no evident public interest isserved by the requirements.

Other respondents also implied that the controlfunction inhibits innovation:

All your questions triggered immediate recogni-tion. Particularly the danger of constraining newdevelopments and fixing a national curriculum inconcrete. (R18, UK, psychology)

At present any innovations I make, which I see aspositive, I must always share but not in a construc-tive manner...more in an ‘asking permission’ typesituation. I can see that this might restrict otherswho may see that they must continue to comply in amiddle-of-the-road fashion. (R37, UK, education)

In seeking permission the last respondent denotes aprocess of supplication. However, the second-orderconnotation is a lack of trust of the academic andthe underlying Barthesian, third order, ‘myth’ isthat there is a body that indeed has the knowledgeand wisdom to grant permission. An alternativetake is to see professional bodies as disengagedfrom the reality of the higher education setting.

Some respondents cited the variability of thevisiting panel as a reason for inconsistency in ef-fecting innovatory change:

In answer to your question re innovation I thinkthe extra burden of yet more self-justifying repre-sentatives on validation panels does nothing toencourage improvements in teaching and/orlearning. (R27, UK, fine arts valuation)

13

ENQA Workshop Reports

The RIBA visit was always a clubby sort of thingand you were either in or out of the club. The panelcoming to visit us next week is almost entirelymade up of thrusting young professional climbersfrom the south east of England, who have the timeto be on endless RIBA committees. They will havelittle idea of how we are working in the hothouseof education today and almost certainly will neverhave heard of the RAE! They will concentratealmost exclusively on ‘Design’ and ignore the restof the course. (R9, UK, architecture)

A holistic view was provided by a respondent whoargued that the accreditation process was cyclicaland this impacted on the innovative potential ofaccreditation.

The bottom line is that there seems to be a cycle –first the educational process gets behind ‘real life’,if the Body is on the ball it writes a report, gener-ates a new specification, etc. The profession’s edu-cation process catches up, the Body can then stag-nate for a bit, and so on. [Innovation] depends onthe Body and which bit of the cycle you are in.Until 1993, nothing much had changed in medicaleducation for 100 years. A relatively radical doc-ument changed all that. Problem-based learning isalive and well and so are graduate entry courses.But much of that comes from government impera-tives – new medical schools, workforce issues, etc.(R5, UK/Australia, medicine)

Here, the external body is reduced to anything butknowledgeable and wise. Indeed it is seen asessentially pragmatic and propelled into action byhistorical necessity.

In the US, it seems that the real motive force forinnovative change in learning and teaching is notthe accreditation process per se but the potential ofthe consequent assessment of student learning out-comes, which research provides a basis for engag-ing with innovative pedagogy.

Bureaucracy and burden

A recurrent theme was the amount of workinvolved in accreditation. A problem accentuatedby rigidity of requirements, perceived at best as

heavy-handed bureaucracy and at worst as anunnecessary degree of control.

One respondent spelled out the problems in de-tail:

I was in charge of a group of people (we called itan Accreditation Committee) – which included allcourse leaders, admissions tutors, etc, over 20people altogether – who prepared documentationand saw through the IEE Accreditation Panel visitlast May for the whole of the School. We got fullaccreditation of all 28 degrees we offer… Thisexercise cost me personally seven months of hardwork (about 50% of my time) and hundreds ofhours of work of my colleagues. … The process istoo bureaucratic and requires too much documen-tation (although we were able to convince the IEEto accept some information in electronic formonly). It takes too much of the valuable time of theacademics and takes them away from research andteaching…. You might be interested to learn forexample that to get 24/24 in QAA we invested 3man-years of work! The IEE accreditation tookabout 1 man-year, RAE (where we got 5* in both1996 and 2001) similar. (R2, UK, engineering)

When asked how might the bureaucracy bereduced, the respondent answered:

In many ways. For example:

• by accepting information already existing in thedepartment/school in the form already availa-ble. We spent days transferring information andrecasting it in the format required, differentfrom ours.

• by not insisting on producing the ‘progressionof students through the programme’ charts –unless the universities have appropriate soft-ware to do it automatically (it took us a monthof hard work to complete these charts and Idoubt if anyone really looked at them!).

• by accepting information in electronic format.For example our entire operation of the Schoolis on intranet and everything the IEE wantedwas there – but admittedly not in the format theIEE forms expected it, hence a lot of our time

14

ENQA Workshop Reports

was spent on pulling the bits of info out andplugging them into appropriate tables or boxes.

I would go even further by suggesting that accred-itation institutions should NOT insist on any par-ticular format in which the information is submit-ted, but they of course should expect that informa-tion does exist and they should indeed be makingjudgements and assessments whether the way inwhich information is kept is appropriate or not.We feel, for example, that our way is better thanwhat IEE wanted from us, so in that sense we werewasting time by reformatting the content. Cur-rently there is far too much duplication of presen-tation of the same information in many differentformats.

Others, noting the amount of work required, wereless negative in the connotations of their remarks:

Yes valuable – although one has to put up with theinevitable requirements for oodles of paperwork(since we had lots of that, it was not problematic!)(R32, UK, education)

The connotation of ‘oodles’ is benign; someonewho has oodles of food has a joyous surfeit andthis is re-presented in the parenthetical comment.The implication being that, if the process is worth-while, the paperwork requirement is an appropri-ate price to pay.

Going further, one respondent talking of theprocess in biomedical sciences noted:

I cannot think of any alternative procedure thatcould ever be as effective as a one-day intensivelook at the syllabus, facilities (laboratories,library, etc.) and staffing. This is streamlined,mutually beneficial to all concerned, encouragesinnovation, and yet is acceptable to the IBMS andthe Health Professions Council. (R21, UK, Bio-medical Sciences)

Despite the apparent compatibility in the biomedi-cal sciences setting, an issue that annoyed manyrespondents was that of synchronisation betweenexternal agencies:

My difficulties with the current system are the hugeamount of paperwork which is spilling out and the

lack of cohesion regarding validation-type visits. Ioften find the QAA, the NMC [Nursing and Mid-wifery Council] and NAO [National Audit Office]are visiting an institution at the same time butrarely share the same documents! (R47, UK, nurs-ing)

Here I do have strong views. I think the accredita-tions institutions, RAE, QAA, and anyone else sub-jecting the universities to continuous assessmentprocesses should agree once and for all – in con-sultation with universities – the format in whichinformation should be kept and presented for allpurposes. Then it is just the question of pressingappropriate button (literally on the screen) toretrieve information for a particular exercise. Wedo have all necessary information all the time andyet every time an assessment takes place we spendweeks or months preparing the documentation.(R2, UK, engineering)

The lack of synchronisation and incompatible doc-umentation is indicative of the desire for differentagencies to control their corner of the quality andstandards monitoring process and, again, onemight ask whether this is in the public interest orthe monitoring organisations’ self-interest?

However, not all the extra burden was externallyimposed. Sometimes the burden is increased by thequality control processes within institutions. The ar-gument might be that this is the inevitable self-pres-ervation response of institutions subject to increas-ing accreditation demands. Certainly, the proposednew accreditation processes in some Western Euro-pean countries are ludicrously bureaucratic and evi-dence of a lack of trust in academia.

Alliance

Curiously, at first sight, given the tensions exploredabove, academics sometimes make use of the pro-fessional or regulatory body to support their ownends. Knowing the power of accreditation in themarketplace, they ally themselves with the profes-sional body. Sometimes this alliance is used to con-serve existing practices and sometimes to makedemands on institutional resources. Gale (2002)noted that:

15

ENQA Workshop Reports

as the teaching of psychology spread from a hand-ful of old universities to the whole higher educa-tion system, heads have found Society accredita-tion a useful political tool. They have used thethreat of withdrawal of accreditation by the Soci-ety as a means of securing enhanced facilities fortheir undergraduate programmes.

Respondents on both sides of the Atlanticremarked on this:

There are some disadvantages to accreditation. Itis expensive and sometimes accrediting teams willmake recommendations that cause money to beshifted from unaccredited programs to accreditedones so that the accredited ones can retain theiraccreditation. This is an unfortunate consequence.(R34, US, general)

[Programme teams] find professional bodyaccreditation provides a ‘bulwark’ against seniormanagement initiatives to reduce resources (R1,UK, personnel and staff development)

Accreditation is most valued by those who areclosest to not having it (the marginal) and by thosewho know how to use it creatively to conduct inno-vative self-evaluations or to strong-arm funderswith “what the accreditors say we absolutely needto retain accreditation” (R33, US, general)

The apparent curious alliance is resolved relativelyeasily. Not only is this a manipulative ploy basedon academic self-interest, using whatever supportcomes to hand, especially in resource-straightenedtimes, but professional bodies are not mutuallyexclusive of academics. Indeed, sometimes seemto be controlled by them:

Some employers seemed to be critical of theactions of the Engineering professional bodies inraising the academic requirements for full char-tered status, partly to enhance the status of theirprofession (in relation to other professions)…. Forsome employers, the fact that the engineering insti-tutions (i.e. the professional bodies) are dominatedby academics reinforces this emphasis on educa-tional needs rather than the needs of the industry.(Little et al. 2003)

Specialist activity

What emerges from all the responses is thataccreditation is a game for specialists; it is notsomething that engages the majority of staff nor, toany significant extent, exercises the students. Forthe latter, accreditation means the kitemark ratherthan the process. It is about uniformity of curric-ula, as one medical student noted, ‘we all need tobe doing the same syllabus’. Part of the controllingelement of accreditation is that it does not engageeveryone and retains an element of mystification.

Power

Accreditation is a struggle for power and it is not abenign process. Nor does it engage all thoseinvolved. It is also not a pure process of identifyingthose who have met (and continue to meet) mini-mum criteria to join the club. The evidence fromthe UK and North America shows clearly thataccreditation is just one of a raft of ongoing proc-esses that demand accountability and complianceas managerialism continues to bite into academicautonomy and undermine the skills and experienceof educators. Accreditation is yet another layeralongside assessment, audit and other forms ofstandards and output monitoring.

The accreditation-improvement paradox

The quality debate in higher education has, for adecade, attempted to engage with the apparentincompatibility, in practice, of accountabilityfunction of external quality monitoring and thehoped-for improvement function (Vroeijenstijn,1995; Middlehurst & Woodhouse, 1995). This ismirrored in the analyses of voluntary accreditationin the United States. Graham, Trow, & Lyman(1995) argued that the accreditation process is fun-damentally flawed because the process of certifi-cation and assurance to the public of the soundnessof the institution’s practices is incompatible withthe improvement of an institution’s performancebased on its continual assessment and evaluationof its strengths and weaknesses. The certificationfunction invariably overwhelms improvementbecause the process leads to the production of apublic relations document that overstates the insti-

16

ENQA Workshop Reports

tution’s strengths and conceals its weaknesses.This is precisely the opposite of what is needed ifthe improvement function is to be served byaccreditation. More to the point, as Murray (2002)notes, accreditation in the United States has notsatisfactorily persuaded the public that the qualityof the professions is safeguarded.

We should also expect to find that the assurance ofquality in the other learned professions is, liketeaching, beyond the capacity of accreditationitself and that it inevitably entails the mechanismsof licensure, certification, peer review, employ-ment, and so forth. The decisions made about thegranting of employment, the professional license,certificate, merit award and honors, should bebased more on solid evidence of accomplishmentthan on conformity to standards, largely unvali-dated, and established by mere consensus of themembers of the profession.

Although the surge towards accreditation in manyparts of Europe is not being delegated to self-inter-est membership bodies in the main, there remainissues of bureaucratic self-interest. Self-perpetua-tion and a growing desire to control are characteris-tic of all types of quality monitoring agencies, espe-cially those with control remits. Furthermore, as theAmerican experience shows, accreditation is notdistinct from quality issues and there is nothing tosuggest that accreditation will not be wrapped roundwith audit, assessment and other forms of qualityevaluation. As the edifice grows and becomes morespecific and directive, so academic alienationincreases, staff perceives a lack of trust and theirown academic judgement being undermined. Theresultant perception of deskilling and diminution ofautonomy and freedom to make pedagogic deci-sions creates a context of compliance and, ulti-mately, as has been seen in other areas of qualitycontrol, game playing, manipulation and subversionof the process (Barrow, 1999). Improvement is along way down the agenda, if it is really on it at all.

Most frustration is expressed at the loss of con-trol of the pedagogic situation and the potential forimprovement. The positive view of the ILTHEprocess (discussed above) is precisely because it

encourages innovation and reflection and dele-gates control to the academic. Although educatorsmay not be aware of the specific concerns of a pro-fessional practice workplace, practitioners areequally unaware of the learning process. Teachers,if not ‘up-to-date’ understand the principles of theprofessional realm they teach about: it is far fromevident that professionals representing accreditingagencies are so well versed in the principles ofpedagogy.

Conclusion

However, the concern is not so much whetheraccreditation is a benign protector of the publicinterest or a process to sustain the self-interest ofthe accrediting agency. Nor, indeed, whether proc-esses are bureaucratic or restrictive and inhibitinnovation. Important as these are, they are indic-ative of a more deep-seated ideological presump-tion summed up in Jon Haakstad’s (2001) thirdnuance of an ‘abstract notion of a formal authoris-ing power’. Repeatedly we saw references tojumping through hoops, tail wagging dogs, askingpermission and the like. Even one of the strongestsupporters of accreditation, who noted that ‘a one-day intensive look at the syllabus, facilities andstaffing… is streamlined, mutually beneficial to allconcerned and encourages innovation’, made itclear that the process needed to be ‘acceptable’ tothe professional and regulatory bodies.

The underlying, third-level, ‘myth’ is that of theabstract authorising power, which legitimates theaccreditation activity. Yet, although taken forgranted, this ‘myth’ of benign guidance is perpet-uated by the powerful as a control on those whoprovide the education. Accreditation is fundamen-tally about a shift of power from educators to man-agers and bureaucrats. It accentuates the trends al-ready evident in the UK towards ‘delegated ac-countability’ (Harvey & Knight, 1996) but revers-es the delegation trend in most of the rest of the Eu-rope. To understand staff perceptions of accredita-tion, the starting point of this paper, requires a ho-listic view that sets the control function of accred-itation within the wider context of higher educa-tion as a public good. It is necessary to dig beyond

17

ENQA Workshop Reports

the surface legitimations of European unity andconsumerist rhetoric to reveal the power processesand the ideology that legitimates the control func-tion of accreditation. Only then can we approachaccreditation openly and critically.

References

Ayarza Elorza, H., 1993, ‘Quality assurance in LatinAmerica: an overview of university accreditation’,paper presented at the First Biennial Conference andGeneral Conference of the International Network ofQuality Assurance Agencies in Higher Education(INQAAHE), Montreal, Canada, May 24-28, 1993.

Baker, R.L., 2002, ‘Evaluating quality and effective-ness: regional accreditation principles and prac-tices’, The Journal of Academic Librarianship, 28(1),pp. 3–7.

Barrow, 1999, ‘Quality management systems anddramaturgical compliance’, Quality in Higher Educa-tion, 5(1), pp. 27–36.

Barthes, R., [1957] 1970, Mythologies (Paris: Seuil).

Campbell, C., Kanaan, S., Kehm, B., Mockiene, B.,Westerheijden, D. F., & Williams, R., 2000, TheEuropean University: A handbook on institutionalapproaches to strategic management, quality man-agement, European policy and academic recogni-tion. Torino: European Training Foundation.

El-Khawas, E., 1998, ‘Accreditation’s role in qualityassurance in the United States’ Higher EducationManagement, 10(3), 43–56

Gale, T., 2002, ‘A stranglehold on the developmentof psychology?’ The Psychologist, 15(7,) pp. 356–59,July.

Graham, P., Lyman, R., & Trow, M., 1995, Account-ability of colleges and universities: An essay. NewYork: Columbia University.

Haakstad, J., 2001, ‘Accreditation: the new qualityassurance formula? Some reflections as Norway isabout to reform its quality assurance system’, Qualityin Higher Education, pp. 77–82.

Hall, Stuart ([1973] 1980): ‘Encoding/decoding’. InCentre for Contemporary Cultural Studies (Ed.): Cul-ture, Media, Language: Working Papers in CulturalStudies, 1972-79 London: Hutchinson, pp. 128–38.

Harvey, L., 1999, ‘Evaluating the evaluators’ Open-ing keynote of the Fifth Biennial Conference of theInternational Network of Quality Assurance Agenciesin Higher Education, Santiago, Chile, May.

Harvey, L., 2001, ‘The British experience in assess-ing competence’ in Palumbo, C.A. and Banta, T.W.(Eds.), Assessing Student Competence in Accredited

Disciplines: Pioneering approaches to assessment inhigher education, pp. 217–43, Sterling, Virginia, Sty-lus.

Harvey, L., 2002a, ‘The End of Quality?’, Quality inHigher Education, 8(1), pp 5–22.

Harvey, L., 2002b, ‘Quality evaluation: some interna-tional trends’, Norwegian Higher Education Confer-ence, Oslo, 22–23 January, 2002.

Harvey, L. and Knight, P., 1996, TransformingHigher Education. Buckingham, Open UniversityPress and Society for Research into Higher Educa-tion.

Harvey, L. and Mason, S., 1995, The Role of Profes-sional Bodies in Higher education Quality Monitoring.Birmingham: QHE.

International Network of Quality AssuranceAgencies in Higher Education (INQAAHE), 2001a,‘Announcements’, QA, 21, November, p. 19.

International Network of Quality AssuranceAgencies in Higher Education (INQAAHE), 2001b,‘Method of the comparative analysis and differentia-tion of the higher education institutions in Russia,QA, 21, November, pp. 15–17.

Jones, D.P., 2002, Different Perspectives on Infor-mation About Educational Quality: Implications forthe Role of Accreditation. Washington, CHEA Occa-sional Paper, April.

Kristoffersen, D., Sursock, A., & Westerheijden,D. F., 1998, Manual of Quality Assurance: Proce-dures and Practice. Torino: European Training Foun-dation.

Little, B. et al., 2003 Vocational higher education –does it meet employers’ needs? (shortly to be pub-lished by the Learning and Skills DevelopmentAgency);

Lubinescu, E.S., Ratcliff, J.L., Gaffney, M.A., 2001,‘Two continuums collide: Accreditation and assess-ment’, New Directions for Higher Education, 113, pp.5–21.

Middlehurst, R. & Woodhouse, D., 1995, ‘Coherentsystems for external quality assurance’, Quality inHigher Education, 1(3), pp. 257–68.

Murray, F.B., 2001. ‘From consensus standards toevidence for claims: assessment and accreditation inthe case of teacher education’ in Ratcliff, J.L, Lubi-nescu, E.S & Gaffney, M.A (Eds.), 2001, New Direc-tions for Higher Education, How Accreditation Influ-ences Assessment, No. 113 April 2001, New York,Jossey-Bass.

Prøitz, T.S., Stensaker, B. and Harvey, L., 2003,‘Accreditation, standards and diversity: an analysisof EQUIS accreditation reports’, paper presented atthe EAIR Forum, Limerick, August, (revised paperforthcoming).

Stensaker, B., 2003, ‘Trance, transparency, trans-formation: the impact of external quality monitoring

18

ENQA Workshop Reports

in higher education’, Quality in Higher Education 9(2),pp. 151–59.

Sursock, A., 2000, ‘Towards accreditation schemesfor higher education in Europe?’, paper at CRE work-shop, 8–9 Nov. 2000, Vienna.

Vroeijenstijn, A.I., 1995, Improvement and Account-ability, Navigating Between Scylla and Charybdis:Guide for external quality assessment in higher edu-cation. London, Jessica Kingsley.

Westerheijden, D.F., 2001, ‘Ex oriente lux?: nationaland multiple accreditation in Europe after the fall ofthe Wall and after Bologna’, Quality in Higher Educa-tion, pp. 65–76.

1.2 Licence to Kill: About Accreditation Issues and James Bond

Ko Scheele, Inspector, Inspectorate of Education in the Netherlands, [email protected]

Accreditation has become something of a hot topicin higher education in Europe. My former SeniorInspector, Frans Leeuw, once called accreditationa “Licence to Kill”. The James Bond metaphor isparticularly illustrative when reflecting on qualityassurance challenges in higher education. If youlook at publications on this subject in recent years,many issues associated with accreditation areexplainable using titles of Bond films.

From Russia with Love: the origin of European accreditation

The Bologna Declaration was signed in 1999 by 29European ministers of education. They declaredtheir support for creating a European Higher Edu-cation Area by introducing a higher educationmodel consisting of two cycles (Bachelor/Masterstructure) and by strengthening quality assurancein higher education. They expressed the firm wishto expand co-operation in the quality assurancefield by such means as the European Network forQuality Assurance in Higher Education (ENQA)and the European University Association (EUA)and by framing new legislation.

Given the European desire for co-operation, itmay seem logical to introduce a single quality as-

surance system, i.e. accreditation, yet it is not anobvious course of action. The Bologna Declarationpurposely avoided using the word accreditation.The European ministers did not want to pin them-selves down to a single system, and definitely notto one uniform quality system. They chose to stickfirmly to the national competence for educationpolicy and quality assurance. Nevertheless, someEuropean countries are now in the process of intro-ducing or preparing accreditation. They includeGermany, Ireland, Italy, the Netherlands, Norway,Austria, Spain and Belgium (Flanders).

When referring to other accreditation systems,Ministers tend to refer to the United States, as thesystem there has existed for several decades. Inter-estingly they do not refer to existing accreditationsystems in Eastern Europe. Forms of accreditationhave existed for many years in Russia, Hungary,Poland and other countries. The reason was the ex-plosive growth of commercial institutions of high-er education. These are not voluntary quality as-surance systems set up by educational institutions,but the system laid down by the government withthe aim of guaranteeing quality in higher educa-tion. Pivotal to the Eastern European approach isthe achievement of the minimum quality standard.

As a result of the Bologna Declaration, a work-ing group, headed by the CRE (now EUA) pre-pared a report on accreditation. The CRE group in-volved in the project started out with the followingworking definition: “Accreditation is a formal,published statement regarding the quality of an in-stitution or a programme, following a cyclicalevaluation based on agreed standards”.

Having a closer look to the CRE-definition anda couple of new accreditation systems in Europe itseems that they opted for a statutory system with apublic quality mark that shows that education sat-isfies the criteria of basic quality. Viewed in thislight, these accreditation systems appear to berooted more in the East than in the West. But justas in “From Russia with Love”, things are not ex-actly as they seem. In the Eastern European sys-tem, the focus is so much on meeting the basicquality standards that improvements beyond thislevel receive less attention. It is therefore no coin-

19

ENQA Workshop Reports

cidence that international experts recommend giv-ing more attention to this matter.

The World is Not Enough: uniformity at INQAAHE and ENQA

The remarkable thing in Europe is that despite thereason given (in the Bologna Declaration) forgreater harmonisation and synergy, we see count-less national initiatives in which accreditation:

• is defined and perceived differently; • is sometimes a complete system, and sometimes

an addition to an existing evaluation system;• embraces the entire higher education system in

some cases, and in others only new courses; • is sometimes built around the institution, and

sometimes around the programme.

Establishing national systems of accreditationtherefore soon tend to produce divergence, in con-flict with the Bologna Declaration.

Yet despite the great differences in the structur-ing of quality assurance systems, there is neverthe-less substantial convergence. Harvey has shownthat, regardless of whether it concerns an institu-tion or programme evaluation, assessment, audit oraccreditation, there is always the same kind of ap-proach. External evaluation always begins withself-evaluation, peer evaluation (evaluation by im-partial experts, usually from the field of study con-cerned), use of standards, performance indicatorsand public reports. Despite the different nationalinitiatives, there are certainly opportunities for theHigher Education Area envisaged in the BolognaDeclaration.

Comparability of systems is a good measure oftrust, the ideal basis for international mobility, andthat is the priority of the European ministers. Butthere are also some potential disadvantages to thisapproach. Harvey mentions “dramaturgical com-pliance”: quality assurance will become a fixed rit-ual, according to fixed procedures, with quality as-surance appearing to become an end rather than ameans. To ensure that sufficient mutual trust ex-ists, initiatives have been launched for a WorldQuality Register. Leeuw rightly mentions that this

leaves the established ritual intact: this worldstandard is not enough.

The Man with the Golden Gun: bespoke or “one size fits all”?

“One size fits all” sums up the existing quality sys-tem in Europe. Comparability is the prime consid-eration, with the aim of assessing similar coursesor institutions in a single procedure using the sameassessment framework. This same principleapplies in the European accreditation system.

The question is whether this development inquality assurance is in line with developments ineducation. According to Leeuw, we see variety innumerous forms in higher education:

• types of education;• administrative relationships between institu-

tions of higher education and universities;• size of institutions of higher education;• the way education is being modernised, based

on the Bachelor/Master structure;• method of selection, such as the introduction of

a binding study recommendation;• dual programmes or other programmes;• extensive or intensive education;• electronic learning environments.

A conceivable scenario for higher education isatomisation and shopping, which will result inindividualisation of learning processes and learn-ing styles. In this scenario, the student’s personal-ised demand for education is becoming increas-ingly important. Quality assurance is geared to thesubstance of learning, to the offerings available,and accreditation could upset the proper harmoni-sation of supply and demand.

The Higher Education and Training AwardsCouncil (HETAC) in Ireland has opted for promot-ing variety, also among non-university educationproviders. A robust attempt has been made to sus-tain and improve the vocational degree programmein analogy with the United Kingdom and Austral-ia. With quality assurance as one of the key factors.The number one consideration is to assure qualityin each discipline. This starts by ensuring variationof quality assurance in each discipline.

20

ENQA Workshop Reports

The proposed approach starts with graduationcourses and continues to the Masters. This promptsthe question of whether the Bachelors shouldmove in the same direction in due course. In theproposed evaluation, the role of information is de-cisive. As mentioned earlier, the submitted andvalidated information must meet stringent require-ments. The institutions in Ireland do not seem tohave much difficulty in this respect, because inter-nal quality assurance automatically generates thiskind of information. Are these the first signs of atrend towards breaking away from “one size fitsall”? Will off-the-peg quality assurance (WaltherPPK) disappear to make way for the Golden Gunof the tailor-made alternative?

Dr No: peers or more

The prevailing quality assurance model includes alarge role to the impartial experts, or peers. Themodel actually encounters very little resistance.Van Berkel, for example, is highly critical of thesystem of peer assessment, but not of the expertsthemselves: “As long as the courses come throughthe peer assessments, they take the view that theyare working in the right way. Courses are satisfiedif they pass the peer assessments. They receive thequality mark from external experts (...) Panel eval-uation relies heavily on the expertise of the mem-bers of the visiting committee (...). Just as now, thecommittees should consist of experts with broadly-based skills in the subject matter of the course andwith educational competencies.” This is a surpris-ing finding, given that he also states that theassessment system does not touch on the essenceof education. Apparently, it is not due to Dr No –who is, after all, smart enough to deliver the qual-ity required – but rather because of the workingmethod.

Golden Eye: the carpenter’s eye and other pointers for the working method

Van Berkel asserts that the peer assessment proc-ess is increasingly being orchestrated by both thecourses and the external organising authorities.Checklists are a popular means to this end. But he

observes that fundamental relationships betweenthe elements within education are not being exam-ined: “Education is more than the sum of all kindsof quality aspects.” Van Berkel advocates a moreholistic approach, with the expert occupying amore central position. And he is not alone inputting the application of established standardsinto perspective.

Lee Harvey makes mention of an orchestratedgame: “Typically, auditors ’hold court’ in the Uni-versity Senate Room and see a stream of visitors,usually in small groups. These groups are sum-moned early by the university senior managers,briefed before they go in to see the auditors andde-briefed when they come out. The auditors heara story that reflects the formal organisationalprocess. Formal structures, though, are signifi-cantly removed from the reality of the living anddynamic organisation that is the university”.

Leeuw follows Harvey in stating that the fixed,established standards and methods allow perver-sion in the quality assurance system: “Evaluationperformance can inhibit innovation and lead totunnel vision and ossification: organisational pa-ralysis brought about by the system of perform-ance measurement (….). It is reasonable to assumethat no matter how well-intended evaluation activ-ities may be, they can and probably will have un-intended and undesired side effects that jeopardiseperformance and /or quality improvement withinthe evaluated or audited bodies”.

Leeuw offers the following solution: “Experi-mental or quasi-experimental designs for evalua-tion are preferred over other designs, triangulateddata are preferred over single method approachesand longitudinal studies are preferred over singleshot studies. Triangulation includes the use ofmystery guests and unobtrusive measures”. In oth-er words: variation and surprise.

A View to a Kill: the example of the trial accreditation

Even before the Minister published his “Mark ofQuality” policy document suggesting proposedaccreditation in the Netherlands, the idea wasalready under experimentation in higher education.

21

ENQA Workshop Reports

In 2000, a trial with accreditation was carried out incourses for social work and services as well asbusiness economics (the so-called ‘trial accredita-tion’). The protocol stated that the most importantdifference compared with the prevailing system ofpeer assessment was the greater emphasis on theobjectivity of quality checks and determination ofthe degree to which the quality of education satis-fies the defined quality requirements. With this inmind, the protocol put forward standards that havebeen elaborated into 80 verification items. Guide-lines were also laid down for the working method(self-evaluation reports, panels and reports).

CHEPS, which evaluated the trial, expressedcriticism on this point in its evaluation report. Theguidelines and standards led to a cumbersomeprocess: the guidelines were far too much of a goodthing; the costs outweighed the benefits. Viewed inthis light, the CHEPS evaluation seems to be a“view to an overkill”. But the evaluation is not en-tirely negative; there are certainly learning bene-fits, among other things because of objectivity. Thedecisions, for example, were thoroughly motivat-ed.

Similarly, the Inspectorate of Education in theNetherlands carefully worded its opinion in itsmeta evaluation of the trial accreditation. Amongother things, it expressed appreciation for the vastmajority of the performed analyses and clearlyworded opinions. But the Inspectorate also ob-served that the protocol of the trial accreditationfor higher vocational education did not prescribe anumber of matters and that they were consequentlymissing from the reports. An example is the ab-sence of the requirement that the assessmentshould take previous evaluations into account. Asregards the set of tools used, the Inspectorate said:“The Inspectorate does consider expansion of theset of tools to be necessary. Where a regulation isnot possible or desirable, the Inspectorate recom-mends using the instrument of triangulation: ex-pansion of the external material with a view to in-ternal, inter-subjective weightings”. One conclu-sion in any event seems inescapable: the quest forthe Holy Grail of optimum quality assurance ismore about smart systems than about large ones.

Live and Let Die: the report

The proof of the pudding is in the eating. Evenwith a very high quality panel and a perfect work-ing method, the entire exercise will be a failure ifthe evaluation is not reported properly. That is whyin its meta evaluation the Inspectorate laid downstringent requirements for reporting. Not onlymust all quality aspects be covered, but the reportmust also be a complete array of information, anal-ysis and judgement. We know from experiencethat information and judgement are usually setdown properly. The analysis is the critical point.

The West-European accreditation laws appearto assume that accreditation can be introduced “ontop of” peer assessment without this generatingany side-effects. This creates limitations that canbe all the more penetrating as far as accreditationis concerned because there is no improvementprocess (in terms of possibly remedying shortcom-ings) but merely a yes/no decision. There are twopossible consequences of this situation: the legiti-macy of the accreditation could be undermined,particularly if the accreditation agency confines it-self purely to validation of the reports. The secondpotential effect is that peer assessment will bejeopardised, or its nature will change. Institutionsco-operate substantially in peer assessment at thepresent time, among other things or perhaps prima-rily because of the existence of an obvious im-provement mechanism. In the case of direct yes/nodecisions, there will be mounting pressure on insti-tutions to adopt a strategic stance. This will in-crease the risk of reports of declining quality. Itgoes without saying that institutions and possiblyalso the visiting committees will be tempted not todisclose information, especially certain parts of theanalysis.

For Your Eyes Only: disclosure?

One of the trickiest dilemmas of higher educationsystems is the degree of openness and disclosure.Despite all beautifully worded pronouncementslike the Bologna Declaration, the European higher

22

ENQA Workshop Reports

education market is still far from transparent. TheEuropean Higher Education Area is central to theBologna Declaration and requires facilitation ofstudent mobility. Minister Hermans recognisedthis by deciding resolutely that the accreditationsystem had to yield information above basic qual-ity level.

One of the objections of institutions to public re-porting is its misuse for “ranking”. It is an under-standable objection. The quality of higher educa-tion consists of numerous aspects that cannot becovered in a template or routine classification.However, rankings will ultimately be made any-way, on account of the transparency and disclosureof all information. Rankings already take place.They are made by private publishers, not only inthe United States but also in Europe. But the man-ner in which the analyses are edited means thatthey are not always consistent and that the rank-ings are in some cases even misleading.

The methodological problems have not led tothe publications becoming unpopular, however,and the opposite is actually the case. Even moresurprising is that the national agencies have notconsidered it necessary to issue their own accuratepublic information. Why is this? For whose eyesare they afraid?

Tomorrow Never Dies: about infinite improvement

Public reporting is one thing, but action inresponse to it is something completely different.The follow-up in higher education quality assur-ance occurs in several forms: evaluation of theprocess, monitoring of improvement plans, moni-toring of improvement activities and, last but notleast, monitoring of the effects of the improve-ment. So are sanctions taken?

The forms of follow-up action in Europe arescarce. That applies very notably to the impositionof sanctions. Quality assurance in Europe is appar-ently set in the framework of the possibility of im-provement and further improvement. What aboutthe Netherlands? Do we show the players a yellowor perhaps even a red card?

The Living Daylights or Die Another Day?

A warning is in the air! Until 2003 proceedingsexist to give study programmes an official warningin the Netherlands. If the given conditions are notmet in due time, the diploma will then no longer bean official higher education diploma and thecourse will no longer qualify for funding. Studentsfollowing the course will no longer be eligible forstudy grants. This amounts to a severe sanction.The basis for taking such a sanction was a reportabout an observed lack of quality, submitted by theInspectorate to the Minister. Conditions for themeasure are that the education provided by acourse must have been demonstrably ailing over aseries of years and that, after hearing the institutionand/or course, there is no confidence in the seriousshortcomings being rectified or the root causebeing eliminated within a reasonable period (oneyear).

In Bond terms, the Inspectorate report is intend-ed “to scare the living daylights out of her”. Butdoes this result in red cards? Despite the embeddedpicture in the Netherlands and beyond, the finalsanction of definitive termination of funding hasnot yet occurred in the Netherlands, although wehave come very close on a few occasions. In anumber of cases, the institutions drew their ownconclusions and closed down the course; in othercases, they took firm action. This was further stim-ulated by the “impending warning”. Unorthodoxagreements are possible in the Netherlands be-tween the Minister and an institution if “a warningis in the air”. That will be the case if there is:

• a prolonged serious shortcoming; • endorsement by the Institution of the Inspector-

ate’s analysis; • an intention to issue a formal warning (yellow

card or worse); • a substantial and demonstrable improvement by

the Institution; • an agreement that the Inspectorate will re-exam-

ine the course in the near future (for example, insix months’ time).

23

ENQA Workshop Reports

Unless the situation improves substantially in theintervening period, the Minister will continue toissue an official warning, followed by the possiblewithdrawal of accreditation. This amounts to a“Die Another Day” scenario. In three instances ithas produced very fast results. The institutionsconcerned immediately took drastic improvementmeasures. The result was ultimately positive.

Accreditation as a yes/no decision renders sce-narios like this impossible. But the pressure on thesystem to make it possible will increase in myopinion.

The Spy Who Loved Me: stimulatory supervisions

The introduction of accreditation will stimulate thedebate: “Who will accredit the accreditors”?

The supervision of accreditation embraces:

• supervision of the accreditation agency itselfconsists of supervision of such matters as thenegligent performance of duties, the holding ofoutside positions, the overturning of decisionsand supervision of the effectiveness of theagency;

• the degree to which accreditation contributes toachievement of the defined goals;

• the degree of effectiveness of the accreditationsystem.

In the Netherlands, the Inspectorate for Educationhas this supervisory task. A specific role for theInspectorate lies in situations where a course isfound not to meet the standards at the time ofaccreditation, but where the Minister decidesunder special circumstances that it is in the publicinterest for the course to continue to exist. In thatcase, supervision of the required improvementsand similar matters is necessary.

Less prominent, but no less important, are thefollowing tasks:

• judging the quality of education; this takes placeat system level, with the exception of examina-tion of observance of rules prescribed by law;

• promoting the quality of education; this, too, oc-curs at system level;

• reporting on the development of education.

It remains to be seen whether the picture of the“Spy Who loved Me” is an appropriate parallel.But it does appear that more is being done in theway of stimulatory supervision than used to be thecase – through the promotion of quality. It is achallenging prospect for the Inspectorate, becausepromoting quality of education is the bottom line,what it is all about. Quality of the highest level asan objective: Diamonds are forever?

References

H. van Berkel. De visitatie-illusie, een pleidooi vooreen andere benadering van visiteren. Thema, 2000No. 5.

Johan C. Van Bruggen, Jacob P. Scheele & Don F.Westerheijden. To be Continued..., Syntheses andTrends. In: To be Continued..., Follow-up of QualityAssurance in Higher Education. Jacob P. Scheele,Peter A.M. Maassen and Don F. Westerheijden(eds.). Elsevier/De Tijdstroom 1998

C.Campbell and M. van der Wende. InternationalInitiatives and Trends in Quality Assurance for Euro-pean Higher Education. ENQA occasional paper 1.Helsinki, 2000.

Centre for Quality Assurance and Evaluation ofHigher Education, Denmark in cooperation withComité National d’ Evaluation, France. Evaluationof European Higher Education, A Status Report. Pre-pared for the European Commission DG XXII. Sep-tember 1998.

Joint Declaration of the European Ministers of Edu-cation, 9-6-1999 (the Bologna Declaration)

European Training Foundation. Final Report andProject Recommendations. Phare Multi-CountryProgramme in Higher Education, Quality in HigherEducation. November 1998.

L. Goedegebuure et al. CHEPS. Alle begin is moeil-ijk, evaluatie van de proefaccreditering HBO. Ensch-ede, May 2002.

L. Harvey. The End of Quality? Paper presented atE-AIR Porto, September 2001

L. Harvey. Evaluating the evaluators. Paper,INQAAHE Chile, May 1999

HBO-Raad. Protocol proefaccreditering. TheHague, November 1999

Higher Education Training Awards Council(HETAC). Guidelines and criteria for quality assur-ance procedures policy. Dublin, 2002.

The Inspectorate of Education in the Netherlands.Accreditatie beproefd, meta-evaluatie proefaccredi-tatie HBO. De Meern, July 2002.

24

ENQA Workshop Reports

Kwartiermakers accreditatie: Prikkelen, profileren enpresteren. The Hague, September 2001.

F. L. Leeuw. Challenges for national quality assur-ance agencies: a view from the evaluation world.Paper at ENQA conference, March 2003.

F. L. Leeuw. Lessons to be learned from other fieldsof evaluation? Lifelong Learning in Europe, 2001No. 2.

F. L. Leeuw. Variëteit en waarborging van kwaliteit inhet hoger onderwijs. Based on a paper at VUBM,Amersfoort, 28 August 2001.

Ministers of Education European Union. Commu-nique of the Prague Meeting. May 2001.

Ministry of Education, Culture and Science. (Whitepaper). Zoetermeer, December 2000.

Netherlands Accreditation Organisation (NAO).Uitgangspunten accreditatie en accreditatiekaders.The Hague, March 2003.

J.P. Scheele. Accreditation, a national or Europeanchallenge? Based on an introduction at E-Air, theEuropean Higher Education Society. Porto, Septem-ber 2001

J.P. Scheele. Program evaluation, and then? Semi-nar ‘Qualitätssicherung im zuge des Bologna-proz-esses’. Bonn, 7-8 November 2002

J.P. Scheele. The follow up of to be continued. Intro-duction, ENQA-CONFERENCE, Barcelona, 18November 2000.

1.3 Premises to Accreditation: A Minimum Set of Accreditation Requirements

Professor Muzio M. Gola, Politecnico di Torino, Italia, [email protected]

Glossary and principles

Evaluation

Evaluation is widely used to express a judgementon the potentials or on the effects of public actions(e.g. political, economic, investments, planning,infrastructure projects).

A more rigorous definition of evaluation is:

• a cognitive activity aimed at providing a judge-ment on an action

• performed following explicit and clear proce-dures

• with the intention to produce outside effects.

Formative / summative

Evaluation can be formative or summative. If anevaluation has a formative function, it is orientedtowards the improvement of actions, to betterstructure and the processes, and to change what isnot working. Formative evaluation is essentiallybased on the qualitative judgement of experts,even if it depends on data or indicators, and it typ-ically concludes with recommendations. The eval-uator becomes, in some way, a participant or co-responsible in the management of the action.

If the evaluation has a summative function, it isinterested in the accountability, in certification or,in extreme cases, in accreditation. A summativeevaluation usually rests heavily on data and indica-tors, and concludes with affirmations or opinions.The evaluator is neutral, attentive to outcomes.

Essential premises

The evaluation must identify and respect certainessential premises:

1) the mandate of the evaluation• know who will use the evaluation,• know what will be the principal use of the eval-

uation (summative or formative)

2) the primary objectives of who is being evaluated• know to what degree the evaluation must be ori-

ented towards:• internal efficacy: comparison of the results ob-

tained from the programme with the initial ob-jectives

• external efficacy: comparison of the results ob-tained from the programme with the outside re-quirements (economic and social context).

3) the instruments of observation and judgement• know the value system of the organisation im-

plementing the action;• define the indicators that describe the primary

objectives coherent with the value system;• know how to concretely gather the information

that will enable us to draw conclusions and ex-

25

ENQA Workshop Reports

press judgements (formative, summative ormixed) based on facts.

Quality

Quality in University formation concerns, obvi-ously, the calibre of the results of the teaching andlearning process.

This definition reveals its difficulties when wetry to define the system of values and the relativeindicators that “bite” into the problem of quality:the competence of the teachers, the suitability ofthe facilities, the existence of an organisation ableto control and intervene in the formative process,the acquisition of knowledge by the students, theirgood results in exams, their pass rate etc.

The ISO 9001 definition of Quality: “the totalityof features and characteristics of a product or serv-ice that bear on its ability to satisfy stated or im-plied needs” in higher education can be interpretedas (Sparkes 1999)2:

“specifying worthwhile learning goals and ena-bling students to achieve them”.

Where:

i) specifying worthwhile goals involves payingattention to academic standards, to the expecta-tions of society, to students’ aspirations, to thedemands of industry and other employers, to therequirements of professional institutions, to thefundamental principles of the subject, etc.; the“stated or implied needs” of these stakeholdersare not all mutually compatible, so there can bemany possible and valid interpretations of “worth-while”.

ii) enabling students to achieve these goalsinvolves making use of research into how studentslearn, adopting good course design proceduresand building on successful teaching experience,all of which may require professional developmentfor most lecturers.

The concept of “fitness for purpose” cannot lead toacceptance of any system that operates accordingto any identified and declared purpose: “fitness forpurpose” must be complemented with “fitness ofpurpose”, i.e., the relevance of the purpose must bechallenged (Kristoffersen, Sursock, Westerheiden,1998). Such complement is guaranteed by dueconsideration of customer needs and requirements.

Accreditation

According to (Hämäläinen et al, 2001)3, the termaccreditation expresses the abstract notion of aformal authorising power, acting through officialdecisions on the approval of institutions (or not) orstudy programmes.

However, if the provider of the accreditation isa public organisation allotting funds, the meaningbecomes quite precise: accreditation is a processaimed at introducing standards of quality, accord-ing to objective parameters, for those subjects whoimplement actions in the formation system in orderto realise public policies for the development ofhuman resources.

Accreditation is a binary judgement (pass – notpass) on the award of a status or on an approval.

It is a process, primarily an outcome of the eval-uation. It can be considered an extreme case ofsummative judgement after an evaluation process.

Responsibility

Responsibility for the quality of the formation is tobe sought at the level where competences aggre-gate and are coordinated, that is, at the level of theprogramme.

The programme has the primary responsibilityfor establishing:

• the professional figure to be trained (integrationbetween the university system and society orwork market),

• the consequent learning objectives (expectedlevel of knowledge and skill that the student

2. Sparkes J.J., 1999, A proposal for a formalised procedurefor achieving good quality teaching of engineering in Eu-ropean universities, Position paper on Quality and QualityAssurance, H3E – Higher Engineering Education for Eu-rope, Working Group n.2 “Quality and Recognition in En-gineering Education”.

3. Hämäläinen K., Haakstad J., Kangasniemi J., LindebergT., Sjölund M., 2001, Quality Assurance in the NordicHigher Education – accreditation-like practices, EuropeanNetwork for Quality Assurance, in Higher Education, Hel-sinki.

26

ENQA Workshop Reports

must have acquired at the end of his studies,foreseen areas of competence and professionalplacement, possible national and internationalbenchmarking),

• the timing, starting from which prerequisitesand with which resources these objectives are tobe reached.

Responsibility in action

It is up to the programme:

• to verify the correspondence between the pro-fessional figure actually produced and the gen-eral prospects of the work market,

• to implement instruments to verify the goodprogression of the teaching programme (studentprogression in quantity, quality and time),

• to coordinate the different formative experienc-es, entrusted to the single teachers in the mostvaried forms (lessons, exercises, seminars,projects, field experience, etc.), check the coher-ence between these and against the objectives,ascertain the compatibility with the study timingand the available resources (human and materi-al).

Transfer of responsibility

Through these acts, documented in a reliable andverifiable manner, the programme provides thereference institution (Faculty, University) with theelements for judgement that will enable them toassume, with an adequate degree of confidence,the final responsibility:

• for the coherence of the study degree with theprofessional figure to be formed,

• for the level of the titles conferred in its name(the effective knowledge and abilities of thegraduating student),

• for the quality of the training provided to enablethe students to reach that level.

Data, judgements and procedures

There are basically three types of instruments, onwhich an evaluation/accreditation model is based:quantitative indicators, qualitative judgements ofexperts and organisation system.

An effective evaluation model must resort to acombination these three “types”. Moreover, it isgeneral practice that it also includes a significantcontrol element such as the gathering of the opin-ions of the students.

• Quantitative indicators

In the grammar of evaluation, these are like sylla-bles or, at most, like words.

Some quantitative indicators of “performance”are essential. It is appropriate that these be pro-duced at a central level in a uniform and certifiedway and supplied to the structures to be evaluatedor accredited. As they are of a numeric type, theycan provide (with due caution for the case) scalesor comparisons of a type that are generally per-ceived as “objective”.

They must be collected, processed, correlatedand compared in a professional way: in developinga set of indicators, the aim is to find a balance be-tween measurability and relevance for drawingconclusions and making judgements.

• Qualitative judgements of experts

In the evaluation syntax these are the sentencesthat make up the discourse.

Many aspects important for the quality of form-ative processes cannot, partly or entirely, be con-veyed as numbers (e.g. the appropriateness of theobjectives or resources, the effectiveness of meth-ods, the results of learning). It is, therefore, neces-sary to have the professional judgement of experts,usually well-known researchers, professors andprofessionals.

These experts benefit from maximum creditwhen they analyse situations that fall into the areaof their direct competence. Their evaluation, insuch cases is constructive and supports the qualitymanagement of the programme.

On the other hand, this process is not so appro-priate for setting up scales of comparison, beingalso slow and costly, and thus it can be replicatedonly in long periods (for example, every fiveyears).

27

ENQA Workshop Reports

• Organisation System

In this case, the question is whether the system iskept under control in an appropriate way. Theresults are not directly evaluated, the implicitassumption being that correct management willbring into play all of the control elements that leadto an analysis, shed to light on the weak points, andtherefore press forward towards the improvementof the results.

As it is a standardised type of evaluation, it iseasier to find experts able to conduct it. But suchexperts can be deceived on the real nature of whatthey are examining and, on the other hand, concen-trating on purely procedural aspects, they run therisk of wasting time with factors that are not strict-ly pertinent to the qualities that are perceived assuch by the students and academics.

Criteria for evaluation and accreditation: a proposal for the debate.

Universities can be very different, not only fromone country to the next, but also among differentscientific sectors within the same country.

In addition, the needs of the three levels of high-er education are different. The three levels of high-er education call for evaluation models based ondifferent approaches.

Level I (bachelor or equivalent) requires astrong emphasis on the legibility of the curriculum(in terms of basic, characterising culture, knowl-edge and skills, target levels, areas of competenceand professional roles envisaged, national and in-ternational benchmarking, if applicable) and on or-ganisational aspects.

The evaluation of level III (Doctorate) should bebased on the ability to provide a markedly re-search-oriented learning environment. It is closelyinterconnected with the evaluation of the researchactivities of the departments. The evaluation oflevel II (Master or equivalent) must take into ac-count the fact that learning contents are geared tothe highly specific (professional or research) goalsof the reference departments. A sizeable majorityof international student exchange activities shouldbe concentrated at this level.

This means that the evaluation objectives andcriteria which are well diversified but share a com-mon requirement and that formulating a finaljudgement on each course of study is based on avery narrow final set of key quality aspects.

The latter should be selected so that, in a clearand readily recognisable manner, they go to thevery “heart” of the quality of educational activi-ties, which are limited neither to the quality of in-dividual teachers nor to formal managerial proce-dures, but rather are the overall quality of a an or-ganised collective effort encompassing severalfronts.

After the review of the general principles, weshould now try to pinpoint a “minimum set” of de-sirable characteristics that should be present in theevaluation models of level I and level II pro-grammes.

Identifying the “minimum set” of evaluation re-quirements suitable for programmes of the firstand second level, common to all countries and toall scientific sectors, appears to be a reasonableand achievable objective. Such “minimum set”could stimulate discussion about what constitutesgood quality within higher education and supportthe development of a common methodologicalframework and common quality criteria for com-parative international evaluations within highereducation programmes.

Basic policy of a programme

A programme should be evaluated on the basis ofits ability to put into effect a policy, focusing –clearly and distinctly – on the external and internal“efficacy” of the learning process:

• specify worthwhile learning goals,• enable most students to achieve the established

objectives.

According to a policy of this sort, quality must beinterpreted in terms of:

• relevance of the purpose (fitness of purpose),• fitness for purpose,

with a special accent on “transformation”.

28

ENQA Workshop Reports

The “efficiency” criterion should be seen as aconstraint affecting the implementation of the pol-icy, not as a policy in itself; therefore, not as an ob-ject of evaluation for the purposes of accreditation.

The mandate of the evaluation

The first and foremost purpose of the evaluation isto reflect the design and management of a pro-gramme: the evaluation checklist should expressthe set of minimum aspects, and the main factorsthereof, that the programme should use before it issubmitted to an external evaluation. The lattershall be conducted on the basis of the same check-list.

The self-evaluation document, as reviewed andcommented on by external evaluators, shall beused by:

• the management of the programme, with aformative and summative function relating to allthe individual actions that put the policy into ef-fect,

• the university that has entrusted the programmewith the task of conferring in its name an aca-demic degree corresponding to effective qualifi-cations,

• government bodies and third parties for the cor-respondence between the qualifications and theacademic degree,

• partner universities, in our particular case thoseincluded in the European circuit, for purposes ofmutual recognition, in particular within thecountries signatories of the Bologna declara-tion,

• all interested parties: by facilitating academicand professional recognition, by promoting in-formed judgements about qualifications that canbe understood in another educational context,by promoting the employability of graduates atnational and international level. (see DiplomaSupplement and Lisbon Convention4).

Vision is needed: policies for evaluation andaccreditation should not remain scaled down tolocal perspectives and to threshold requirements.

The focus of the judgement

The instruments of the external evaluation are:

• indicators with summative functions: in particu-lar, indicators of intake, progression, success ofthe student and of the graduates,

• experts’ judgements with both summative andformative functions, on the aspects and factorsrequired by the model.

The organisational system, which is highly varia-ble from one case to another and is always devel-oped over several levels (programme, faculty, uni-versity), should be left in a free format and shouldbe evaluated ex-post, in terms of its suitability tosupport those actions having a bearing on the inter-nal and external efficacy of the programme.

Thus, it is sufficient to ensure that the followingindications are provided for each aspect / factor en-visaged by the model:

• it must be absolutely clear which person or com-mittee is responsible for the policy, the qualityand the execution of all educational matters re-lating to a given study programme,

• that those responsible discharge their dutiescompetently and on time,

• that each action is documented in a pertinentand accessible manner.

In other words, that the effectiveness of an organi-sational system is evidenced by the description ofthe actions and their documented effects, factor byfactor.

Changing the philosophy of the self-evaluation report

Our proposal is to discard the logic and practice ofperiodic “evaluation reports” and adopt a logic ofon-going monitoring: it is desirable that each pro-gramme be required to keep and regularly adjournan “information model” that collects and updatesthe quantitative parameters and the qualitativedescriptions enabling other parties (e.g. academic

4. Council of Europe and UNESCO “The Convention on theRecognition of Qualifications concerning Higher Educa-tion in the European Region” Lisbon 8–11 April 1997.

29

ENQA Workshop Reports

authorities, third parties and external evaluators) toformulate an informed judgement.

This “information model”, which preferablyshould be made fully known to the public, can beflanked by a “self-evaluation supplement”, reservedto evaluation authorities, discussing the strengthsand weaknesses of the programme.

The structure of the information model

The different items to be considered in evaluatinga programme can be grouped into four key“aspects” or “dimensions” of the evaluation:

• Requirements and objectives• Teaching, learning and assessment• Learning resources• Monitoring, analysis, review

An appropriate quality assurance mechanism willbe present if these four aspects are managed effec-tively by the programme.

Each “aspect” is clarified through a certainnumber of “factors” separately indicated, eventhough it will be of great value to consider their in-terconnections (e.g., the structure and content ofthe programme must be described as a logical ex-pansion of ensuing general educational objectives,benchmarking).

The contents of the information model

Let us examine the most critical factors.

• Requirements

The first aspect of the model is “Requirements andobjectives”. In order to determine the require-ments, it is necessary to identify clearly the partiesconcerned.

In some instances, it is possible to stipulate averitable alliance with the world outside the uni-versity as a valuable aid to overcome deep-seatedhabits and to increase public awareness of the logicunderlying the programme.

• Educational Objectives

The translation of the “requirements” factor into“educational objectives” is performed by the uni-versity; it uses the know-how and the language of

training specialists; it consists essentially of har-monising the knowledge building processes andlearning outcomes that meet the requirements.

This is the point at which it is necessary to re-flect critically on the strategies, make choices, andclearly express justifications for the chosen priori-ties.

The best guide currently available for the for-mulation of learning outcomes is provided in the“Benchmarking Statements” by the QAA. Thisdocument could perhaps be adopted as the startingpoint for the definition of educational objectives,in terms of contents and levels.

• Teaching, Assessment Methods

Once the educational objectives of the programmehave been identified and deployed as specificobjectives of the individual courses of study, theteacher is provided with great freedom of action asto the methods to be employed in order to achievethem and to ascertain whether they have beenachieved.

The teacher and his/her course of study repre-sent a complex system, whose management re-quires competencies of a technical-scientific na-ture as well as pedagogic and social competencies.

Effective system operation hinges on a diffusedpropensity to reflect, i.e., the ability of each teach-er to observe the effects of his/her actions and tomake appropriate corrections, as necessary.

People are the fundamental element in the qual-ity of services, especially those involving a highcontent of expertise. But assessing people usingobjective criteria is by definition very difficult, andthis is especially true for professionals in highereducation. It is advisable, however, to preventteachers from proceeding by trial and error. Thiscan be done through specialist training pro-grammes for newly-hired teachers, to enhance thepedagogic and teaching skills they need to managethe classroom and apply the assessment techniquescompetently.

An effective way to assess the behaviour of ateacher ex-post is to verify the contents of the ex-aminations in order to determine the knowledge/skills they are designed to assess, and the marking

30

ENQA Workshop Reports

criteria adopted. In other words, to determinewhether the tests ascertain the presence of theknowledge/skills required (and made known be-forehand), avoiding both false negative and falsepositive results.

The collection of student opinions by means ofquestionnaires or other equally effective ways is acomplementary method that can supply useful in-dications.

Breaking down the “factors” into their constituent “elements”

A working description of the factors is provided bybreaking them down into their “elements”; anoverview of the evaluation modes supplies manyinteresting indications.

A list of common elements helps to make theevaluation reports more comparable; however, it isadvisable to leave freedom of choice in the selec-tion of the elements making up a factor. At this lev-el, a holistic approach stressing the interdepend-ence between the elements and their complemen-tarity should be encouraged.

Accordingly, while, as a rule, it will not be pos-sible to accept compensations between the factorsof an aspect, it is reasonable to consider the possi-bility of compensations between the elements that,taken together, add up to a factor.

Thus, the information model will reveal that theprogramme is much more than a static configura-tion of components or a mere list of actions. In-deed, it is a self-organised structure, susceptible of

evolution and development, to be assessed on thebasis of clear and explicit criteria.

Conclusion

It is desirable that each programme be required tokeep and regularly adjourn a public and on-line“information model” with a view to:

• enabling any competent authority or party to as-certain whether the quality of the qualificationsissued by the institution justifies recognition,

• enabling prospective students and employers toformulate informed judgements about expectedqualifications and means /resources made avail-able by the programme to support the learningprocess.

A programme should be evaluated on the basis ofits ability to put into effect a policy focusing –clearly and distinctly – on the external and internal“efficacy” of the learning process.

An appropriate quality assurance mechanismwill be present if the following four key “aspects”or “dimensions” are first of all described in the “in-formation model” and then kept under control inan effective manner by the programme:

• Requirements and objectives• Teaching, learning and assessment• Learning resources• Monitoring, analysis, review

The “key aspects” and their articulation in “fac-tors” may be the basis of an agreed “minimum set”of requirements for the information model.

31

ENQA Workshop Reports

2 Country Cases

2.1 AUSTRIA: Accreditation in Austrian Fachhochschule Sector

Kurt Sohm, Managing Director, Fachhochschul-erat, [email protected]

1. Austrian FH-sector

The Austrian FH sector has a short history and isstill being developed:

• The Fachhochschule Studies Act became effec-tive on 1 October 1993.

• The first 10 programmes started in the academicyear 1994/95.

• Currently there are 19 institutions offering 144programmes and about 22.000 students (2007:about 28.000).

The whole sector has been completely anewlydeveloped since 1994, combining (public) top-down control and (private) bottom-up initiative →education is not offered by transforming existingeducational institutions but by accrediting newprogrammes.

Framework conditions:

• The course providing bodies are – with two ex-ceptions – privately organized (legal personsunder private law, e.g. companies with limitedliability, associations or public foundations)

• Public funding• External Quality Assurance by the FH Council• The legal control is with the Government.

2. Austrian FH-sector

FH institutions are given greater autonomy toorganize themselves

• Decentralisation and deregulation of decision-making process

• To foster the independence, responsibility andflexibility of the FH institutions

Financing concept of study place management• Costs per study place and year are about 7.600

EURO in technical fields and about 6.400 € inthe business fields

• Federal government pays only 90 % of thestandard costs per study place (6.900 resp.6.400 €)

The Amendment of the FH Studies Act of May2002 facilitated the introduction of bachelor andmaster programmes:

• Duration of study: Bachelor = 3 years (180Credits); Master = 1 to 2 years (60 to 120 cred-its); Diploma = 4 to 5 years (240 to 300 credits)

• The first bachelor programmes started in the ac-ademic year 2003/04; the first master pro-grammes will be started in 2004/05.

3. Austrian FH sector

Educational mandate: practice-oriented profes-sional education at higher education level. Basicconcept of a FH programme can be characterisedas follows

• Description of the vocational fields of activityfor which the programme is aimed;

• The professional field-specific requirementshave to be presented in the form of a qualifica-tion profile;

• This qualification profile forms the basis for thedesign of the curriculum.

The review of the coherence of vocational fields ofactivity, qualification profile and curriculum by aFH degree programme plays a very important rolein the accreditation procedure.

32

ENQA Workshop Reports

4. External quality assurance system

FH Council is the public authority responsible forexternal quality assurance. All FH programmes(Bachelor´s, Master´s and Diploma programmes)are subject to accreditation. Accreditation isgranted for an approval period of a maximum of 5years.

Tight connection exists between decisions oninitial- and re-accreditation of programmes andevaluation of programmes and institutions

• Initial accreditation is compulsory in order toget an approval for a FH programme (accredita-tion is equivalent to approval);

• The decision on initial accreditation is carriedout by the members of the FH Council theme-selves;

• Each decision of the FH Council on the re-ac-creditation of programmes is based on a previ-ously conducted evaluation.

5. Accreditation in the national context

Evaluation = quality improvement? ↔ Accredita-tion = quality control?

No → Accreditation = Evaluation + decision onquality + approval

Accreditation

• is a formal and independent decision, indicatingthat a program offered and/or an Higher Educa-tion Institution is meeting certain standards;

• is based on a previously conducted evaluationprocedure that estimates the value or benefit ofmeasures with respect to the compliance withcertain standards;

• includes quality improvement according to theevaluation results;

• ends with a positive or negative decision.

Aim of accreditation

• to assure that the institutions meet their respon-sibility for the quality of the programmes of-fered;

• to guarantee students, society and employersthat the programme has to undergo a quality as-surance procedure before it is approved or re-approved.

6. Initial accreditation

Programmes are designed on behalf of the courseproviding bodies by expert teams with the requiredacademic and professional qualifications. Anapplication for initial accreditation/approval of aprogramme is submitted to the FH Council accord-ing to the “accreditation guidelines”.

The applications are examined by the office ofthe FH Council and by the members of the FHCouncil in plenary meetings.

• if there is no expertise in the FH Council, writ-ten expert opinions are asked for;

• the accreditation procedure has to ensure thatthe legal prerequisites are met;

• requests by the FH Council to take certain meas-ures to improve the quality of the programmeare part of the procedure.

Since 1994, about 40 % of the applications havebeen rejected by the FH Council. The most impor-tant question that needs to be answered positivelyfor an (initial) accreditation is whether the pro-gramme is able to fulfil its educational mandate ina reliable and transparent way.

7. Re-accreditation

Fields of institutional and degree programme-related evaluation

No Fields of institutional evaluation

No Fields of degree programme-related evaluation

1 Strategy and organi-sation

1 Educational goals and teaching methods

2 Quality manage-ment and HR devel-opment

2 Students

3 Degree programmes

3 Organisation and quality assurance

4 Students 4 Human resources

5 Applied research & development

5 Infrastructure and ap-plied research & development

6 Resources, infra-structure and funds

7 Internationalisation, co-operation and communication

33

ENQA Workshop Reports

The aim of self-evaluation

To show in a transparent, well-founded and relia-ble way how the aims, requirements and expecta-tions as defined in the fields to be evaluated aremet.

The aim of external evaluation

To evaluate, on the basis of the self-evaluation,whether the aims, requirements and expectationsas defined in the fields to be evaluated have beenconvincingly and transparently fulfilled.

8. Re-accreditation

Examples of institutional evaluation

9. Re-accreditation

Examples of programme-related evaluation

8.1 Strategy and organisation

8.1.1 The FH institution has a clearly formulated stra-tegic orientation that has been put forth in a public mission statement. It sees itself as a learning organisation and ensures the advance-ment of the institution based on its strategic ori-entation.

8.1.2 The mission statement lays down the educa-tional and research goals and serves to position the institution within the academic and social environment. The desired goals are appropriate-ly communicated within the institution.

8.1.3 The decision-making processes, competences and responsibilities are clearly defined, commu-nicated and implemented. The implemented or-ganisational structure and procedure ensures the institution’s autonomy and is critically re-viewed regarding its efficiency and effective-ness.

8.1.4 The organisational structure and procedure en-sures that the faculty is integrated in the deci-sion-making processes related to study and re-search and that the students are integrated in decision-making processes related to educa-tion. The faculty’s autonomy is ensured in a way that is appropriate for a higher-education institu-tion.

8.1.5 The creation or promotion of a corporate identi-ty is ensured. Based on the exchange of knowl-edge between the experts in the fields of teach-ing, research, business, administration, etc. who work in the FH sector, the process of knowledge management is organised.

9.1 Educational goals and teaching methods

9.1.1 The educational goals have been clearly defined and everybody involved is familiar with them. The connection between the vocational activi-ties, the qualification profile, the curriculum and the teaching concept has been described in as conclusive way and documented transparently.

9.1.2 Placements form an integral part of the curricu-lum. The educational goal of the placement has been defined and everybody who is involved ei-ther in the institution or in the company is famil-iar with it.

9.1.3 The process of selecting, qualifying, tutoring and assessing the placements is defined and implemented accordingly. The working relation-ship between the FH institution, the company and the students has been defined in a contract.

9.1.4 The knowledge and skills to be acquired in an FH degree programme based on the vocational and higher-education requirements are docu-mented in a sufficient and transparent way with-in the scope of a qualification profile.

9.1.5 The curriculum provides the relevant scientific knowledge and understanding, the methodolog-ical-analytical skills, as well as the multidiscipli-nary qualifications to be able to meet the aims related to the vocational field in a way that is ad-equate for a higher education institution.

9.1.6 The selection of the related disciplines is justi-fied and the relation of the respective discipline to the desired vocational field is described.

9.1.7 The contents of the curriculum as well as the teaching methods used to implement it are suit-able for reaching the educational goals set forth. The proportion of the different types of courses (lectures, training courses, seminars, place-ments, projects, etc.) is balanced with regard to the educational goals. The acquisition of active competences typical of higher-education insti-tutions and vocational competences is encour-aged.

34

ENQA Workshop Reports

2.2 FINLAND: Accreditation Models in Higher Education in Finland: Experiences and Perspectives

Kirsi Mustonen, Senior Adviser, Finnish Higher Education Evaluation Council, [email protected] Moitus, Project Manager, Finnish Higher Education Evaluation Council, [email protected]

The Finnish Higher Education Evaluation Council

The Finnish Higher Education Evaluation Council(FINHEEC) is an independent expert body assist-ing universities, polytechnics and the Ministry ofEducation in matters relating to evaluation. Thescope of the activities covers 20 university levelinstitutions and 29 polytechnics.

The main objective of FINHEEC is long-termdevelopment of higher education through evalua-tion. The main duty of the Council is to assist high-er education institutions and the Ministry of Edu-cation in evaluations, and to develop evaluationprocedures in higher education institutions nation-wide. Consequently, the Council strongly empha-sizes the role of the higher education institutions inevaluations as well as a communicative evaluationapproach in its evaluation projects.

FINHEEC is appointed by the Ministry of Edu-cation for a four-year period. The duties of FIN-HEEC are based on a Decree (1320/1995), whichstipulates the duties to the Council:

• Assisting institutions of higher education andthe Ministry of Education.

• Conducting evaluation for the accreditation ofthe polytechnics.

• Organising evaluations of the activities of high-er education institutions and evaluations relatedto higher education policy.

• Initiating evaluations of higher education andpromote their development.

• Engaging in international co-operation in evalu-ation.

• Promoting research on evaluation of higher ed-ucation, and

• Evaluation and acceptance of professionalcourses offered by higher education institutions,entering of courses into a register stipulated inArticle 14 of the Decree on the Higher Educa-tion System and maintaining such a register(Decree 456/98).

The types of evaluations conducted by FINHEECcan be categorised as follows:

1. Evaluations of official nature• accreditation of polytechnic operating licences • accreditation of professional courses offered by

higher education institutions• evaluation of applications to award polytechnic

post-graduate degrees

2. Evaluations initiated by FINHEEC• evaluations of higher education institutions: in-

stitutional evaluations, audits of quality work• programme and thematic evaluations

3. Evaluations commissioned by the Ministry ofEducation

• selection of the Centres of Excellence in Educa-tion and Adult Education in the university sectorand Centres of excellence in Education and Re-gional Impact in the polytechnic sector to beused in performance-based appropriations.

The introduction of accreditation into the highereducation sector in Finland is one element in thenational quality assurance system. However,enhancement and assessment of the quality of edu-cation has so far been seen as more important thanaccreditation. Two models, illustrating how FIN-HEEC is involved in accreditation-like practices,will be discussed in the text below.

Accreditation of Professional Courses in Higher Education Institutions

In 1998, FINHEEC was assigned the task ofassessing and registering professional coursesorganised by higher education institutions. Theterm “accreditation of professional courses” iscommonly used about this type of evaluation. TheMinistry of Education appointed a subsection, the

35

ENQA Workshop Reports

Accreditation Board of Professional Courses,whose task is to assess professional courses (incontinuing education) of at least 20 credits by vol-ume and make the decisions of accreditation. TheBoard consists of 12 members, who represent uni-versities, polytechnics, working life and studentunions, of both the university and polytechnic sec-tors.

Accreditation of professional courses is a proc-ess that gives public recognition or registering toprofessional, non-degree courses meeting certainstandards/criteria. The accreditation process in-cludes a review of relevant documentation (appli-cation), a visit to the university/polytechnic andthe immediate feedback after the site visit. TheBoard has seen it necessary to also use external ex-perts in the process. However, the final decision isdone by the Board. When accrediting professionalcourses, the Board has set the criteria for goodpractices. The decision is usually done on a yes/no(registered/not registered) basis. Sometimes thedecision for registration can be conditional. In thiscase, the applicants are given a certain time period(1–2 years), during which the needed additions oramendments have to be carried out. Evaluatedstudies are always assessed in terms of the criteria,not for example of other studies of the same field.Feedback and recommendations for the course areprovided after the registration decision is made.

Accreditation of Professional Courses includesthe following steps:

• An application form, designed for accreditationpurposes, is available for the applicants (highereducation institutions) on FINHEEC’s website(http://www.finheec.fi/erikoistumisopinnot).The institution fills in the application and ap-plies for the accreditation on a voluntary basis.

• After receiving the application, the Accredita-tion Board appoints two (or three) external ex-perts to conduct a site visit to the higher educa-tion institution. During the visit they discusswith different stakeholder groups. Typically theteam consists of two members, an expert on thecontents of the professional course and a senioradviser from FINHEEC, acting as a pedagogicalexpert.

• During the site-visit to the organiser of thecourse, the following aspects are analysed:– Basic requirements– Course contents and objectives– Educational process– Educational arrangements– Practical arrangements– Cooperation with the working life– Quality assurance

• After the site visit the experts make a proposalfor acceptance (yes/no) to the AccreditationBoard.

• The Board uses the evaluation criteria to makethe final pass/fail decision. The main groups ofthe criteria are: requirements, work-orientation,contents and objectives, the educational proc-ess, pedagogical arrangements, practical ar-rangements and quality assurance. Special crite-ria are used for the professional courses taughtin a foreign language.

• Higher education institutions receive the deci-sion within four months of the delivery of theapplication. Together with the decision a writtenfeedback with strengths and developmental as-pects of the professional course is given to theapplicant.

• The registration of accredited courses is validfor 4 years. The register is available on FIN-HEEC’s web-site.

• A separate form should be filled for each profes-sional course applying for re-registration.

• The decision by the Board costs 1005 € for theorganiser of the course.

There are 72 courses/applications that have beenevaluated by the Board in the years 1998–2003 andthree applications are currently under evaluation.22 applications are from the universities and 53from the polytechnics. More than 50 % (N= 41) ofthe applications were from the field of Health careand Social services. Eight applications were fromthe field of Business Economics (for exampleMBA-programmes) and 5 from Management. 21applications were from different fields of studies,such as Education and Engineering.

About 66 % of the professional courses havebeen approved to be registered. The number of ap-

36

ENQA Workshop Reports

plications is very low compared to the amount ofthe professional courses organised by the universi-ties and polytechnics. For example, in 2002 therewere more than 300 professional courses offeredby the Finnish universities.

Accreditation of Polytechnic Operating Licences

In 1995 the Finnish Government defined the strat-egy for carrying out the polytechnic reform withthe aim to establish the polytechnic system by2000. The goal of the reform was to establish a newkind of degree with practical orientation to educat-ing professionals for expert and development posts,while the basic mission of universities is to carryout research and provide education based on it. Thepolytechnics were formed on the basis of post-sec-ondary vocational institutions by raising theirstandards and by merging several institutions tocreate multi-field polytechnics. In creating the pol-ytechnic system, emphasis was laid on multidisci-plinary, regional institutions, which give particularweight to contacts with business and industry.

The polytechnic reform was based on an exper-imental phase of 1991–1994. Each new polytech-nic had to be preceded by an experimental and de-velopmental stage. The basic assumption was thatlicences for permanent polytechnics would only begranted after they could demonstrate high qualityand good performance during this experimentalstage. The core of the strategy would thus be con-stant development, and gradual attainment of per-manent status.

The Finnish Higher Education EvaluationCouncil assisted the Government in the accredita-tion of the polytechnics. FINHEEC evaluated ap-plications made by the polytechnics for accredita-tion and establishment. A separate AccreditationSubcommittee was established. The Members ofthe Accreditation Subcommittee consist of the rep-resentatives of polytechnics, teachers working inthe polytechnic, students and representatives ofworking life.

Operating licence evaluations were made to en-sure that new polytechnics and the education theyprovide, meet the quality criteria for higher educa-

tion. Each evaluation incorporated proposals formeasures to develop the particular polytechnic’soperations. If an experimental project failed tomeet FINHEEC’s criteria, a new application had tobe submitted the following year. This meant thatevaluation of operating licence applications com-prised a process of great significance for educa-tional development throughout the sector.

In 1995 and 1996, the accreditation and exten-sion of polytechnics were evaluated on the basis ofapplications. Since 1997, site visits have been add-ed to the procedure. The Accreditation Subcom-mittee has compiled public reports of each evalua-tion and, since 1998, these reports have been pub-lished in the FINHEEC publication series.

The criteria used in the accreditation of perma-nent polytechnics mainly include proven excel-lence in experimental and development work. Thefollowing framework for criteria was used in theassessment, of which 1–12 are mentioned in thePolytechnics Act:

1. The operating principle (mission) 2. The topicality and need for the planned degree

programmes 3. How well the sectors of study fit together 4. The main area of strength5. Adequate size relative to educational function6. The qualifications of the teaching staff7. Library and information services8. Relations with the working life9. Cooperation with universities/other polytech-

nics, and with other educational institutions 10.International cooperation11.Educational and service function in the region12.Arrangement of evaluation13.The learning environment14.The working environment

Some of the granted licenses in 1998–1999 includ-ed special development obligations. These werecases when the Government approved a polytech-nic’s licence, although according to the evaluationmade by FINHEEC’s Board, it didn’t measure upto the demanded quality. Accordingly, the licensewas granted, but the set obligations had to be ful-filled and later evaluated by FINHEEC’s Board.

37

ENQA Workshop Reports

This evaluation task formed a major part of thework of the Accreditation Subcommittee in 2000-2003. Polytechnics with special development obli-gations had to give a report on the results of theirdevelopment work, separately of each dimensionof given development tasks. In other respects theevaluation process followed mainly same phasesand principles as that of the previous evaluation ofthe permanent operating licences.

Furthermore, the Accreditation Subcommitteehas implemented evaluations when there has beena change in the scope of activities of an accreditedpolytechnic, or in the event new educational estab-lishments (former independent institutions) havebeen incorporated with it. Since August 2000, all29 Finnish polytechnics have been permanent.

Discussion

Accreditation is seen as one of several comple-mentary measures in the Finnish quality assurancesystem, whose starting point is to support highereducation institutions in the development of goodquality. One special character in FINHEEC’s eval-uations is that accreditation has related only to theevaluations of official nature: accreditation of pro-fessional courses in higher education institutionsand accreditation of operating licences of the pol-ytechnics. The other feature common in accredita-tions applied by FINHEEC has been the integra-tion of a developmental aspect, which is in accord-ance with the FINHEEC principles. There is evi-dence that combining accreditation with develop-mental approach is useful to the higher educationinstitutions when developing their activities.

The accreditation of professional courses is vol-untary to the universities and polytechnics. How-ever, the evaluation, accreditation and registrationof professional courses serve the universities, pol-ytechnics, students, the working life and the socie-ty in general, by aiming to ensure that the regis-tered studies have a certain, generally approvedlevel of quality, a “quality label”. At its best, theaccreditation process can be important tool in de-veloping the quality of the continuing education.

In 2003 the Accreditation Board has done a sur-vey on the impact of the accreditation processes tothe development of professional courses at thehigher education institutions. The questionnaireswere sent to all those universities and polytechnics(N = 64) whose professional courses were evaluat-ed in 1999-2002. 75 % of the institutions replied.According to the survey, the evaluation processand the written feedback has been an importanttool especially in the development of the planningprocess and structure, objectives and contents ofthe curriculum of the professional course. Therealso seems to be a positive transfer impact to thedevelopment process of other professional coursesin the institution than simply to those been accred-ited. Additionally, according to the survey the pol-ytechnic sector has used the accreditation of pro-fessional courses to strengthen the experimenta-tion of polytechnic post-graduate degrees. Theprofessional courses can be partly integrated to thepost-graduate degrees of polytechnics. The regis-tration period (4 years) of the professional coursesthat were accredited in 1999 will end in 2003. It isnow time for a re-accreditation process, which issimilar to the first-phase accreditation.

The type of accreditation applied in the evalua-tions for the accreditation of the polytechnics rep-resents quality assurance that is implemented exante, that is, before the start of the polytechnic’seducational activities. Developmental aspect inthis process was realised in two ways: the institu-tions applying for a polytechnic operating licensereceived a feedback consisting of strengths and ar-eas to be developed. Then a follow-up was organ-ised. Many institutions worked for 2–3 years toreach the standards set for a polytechnic status.Now that this unique accreditation process is com-pleted and the polytechnic sector is established, itcan be estimated that the process has been useful tothe polytechnics. Two very important results of theaccreditation of polytechnic operating licences, theculture of quality assurance and a strive for contin-uing development, have emerged in the polytech-nics.

38

ENQA Workshop Reports

2.3 FRANCE: Evaluation and Accreditation Practices in France. A Powerful Tool for Improving Quality in Engineering Education

René-Paul Martin, Commission des titres d’in-génieur, www.commission-cti.fr

Our topic is the evaluation and accreditation ofengineering curricula, specifically arising from theyears of experience of French accreditation inengineering (Commission des Titres d’Ingénieur –CTI). The CTI exerts its activity from outside theinstitutions (Grandes écoles, university engineer-ing schools) and has defined a set of references,relevant evaluation criteria and techniques neces-sary for the accreditation process. The objectivesare to:

1. Modernize and internationalize of engineeringtraining;

2. Close the gap between ‘products’ and ‘needs’by a strong involvement of industry;

3. Develop a constantly reviewed accreditationsystem;

4. Search for excellence to attract the elite.

Quality management can be a means to improvethe quality of teaching and learning, and the eval-uation process by external body with a view toaccreditation.

The Commission des Titres d’Ingénieur was ap-pointed by an Act of 1934. Its aim is to accreditatethe Ecoles d’Ingénieurs and to award the title of‘Ingénieur diplômé’. Total education and trainingduration is five years and equivalency is given to aMaster degree. The Ministries of Education, In-dustry, Defence or Agriculture subsequently takeup CTI’s recommendations for their public institu-tions and attribute the budget and personnel. Pri-vate institutes make their own decisions.

The CTI has 32 members. Half of these hold po-sitions in higher education or are members select-ed for their scientific or technical experience. The

other half represents the various aspects of the pro-fession. This twin composition is particularly in-teresting as it brings together engineering trainerswith experts, representatives of the professionsand of industry as well as representatives of themain trade union organizations and engineeringassociations. Members are appointed for 4 years,which is renewable once. The CTI meets 11 timesa year in plenary sessions.

Any public or private institution that has a high-er level course of training in science or technologyfor specific vocational purposes can apply for thecourse to be awarded the title of “ingénieurdiplômé”.

The Commission then appoints reportersamongst its members and possibly also experts. Amission is organized to study the teaching pro-gram, meet the management and teaching teams,the students, employers and former students, andto inspect the teaching facilities and laboratories.The report is presented to the CTI, who pronounc-es its opinion or decision (private institutions).

Following an experimental phase in 1990–1995,general recognition for a fixed term of maximum 6years was established in 1997. The CTI is empow-ered to intervene at its own initiative. The coursesin the 250 institutes, which together grant 30 000degrees per year, are periodically evaluated.

The engineer: job, experience and training

What does an engineer do?

Essentially the job of an engineer is to identify andfind solutions to problems of a concrete and oftencomplex technological nature related to the design,realization and use of products, systems or serv-ices. This aptitude arises from an ensemble oftechnical knowledge on the one hand and eco-nomic, social and human experience on the otherhand, with a basis in sound scientific training.

The main areas of activity for engineers are inindustry, construction, agriculture and the serviceindustries. This activity mobilizes human, techni-cal and financial resources, usually in an interna-tional context. It receives economic and socialsanctions and is concerned with protecting man,life, the environment and the collective well being.

39

ENQA Workshop Reports

What training does an engineer receive?

Training does not just involve mastery of a tool ora computerized approach to problems. The notionof technological purpose has been changed by theincreasing importance of computerized simula-tion, which in turn leads to the need for a high levelof skill in modelling and optimization techniques.

Programs

• Course content

An engineer requires multidisciplinary training,proof of his/her ability to handle the various mis-sions with which he/she will be entrusted duringthe course of his/her career. The training courseshould include:

• Detailed teaching of basic science that may wellinclude a first research experience

• Full training in general engineering techniques,including mastery of complex systems.

• A sufficiently long course in the main aspects ofthe desired training

• General education: foreign languages; econom-ic, social and human sciences, a concrete ap-proach to problems, communication, ethical is-sues, social relationships in all contexts

• Training for the industrial environment and theconstraints involved, in particular with regard tothe environment, safety, health, quality, globali-zation, and industrial property.

• Duration

• An initial training of three years of 2 200 to 2700 supervised hours – taught classes – directedstudy – practical tasks and projects.

• By initial apprenticeship: 1 800 hours plus 6months per year in a company – 3 years (plus 2years external or integrated experience and sci-entific higher education).

• By in-house training: 2000 hours formal teach-ing; professional experience is taken into ac-count.

In-company placements (internships) as part of the training of engineers.

The CTI regards giving student engineers anawareness of industry as an essential part of thistype of training. It is important that a trainingcourse should have a well-balanced content:advancement in knowledge and relevance to thestructure of the industry and the job in question.The accumulated length is between 20 and 36weeks.

International awareness

Multicultural awareness is a basic requirement forthe practice of engineering skills. This means thatthe curriculum should include, alongside the prac-tice of foreign languages, the development of insti-tutional exchanges between France and othercountries. The CTI also encourages growth in thenumber of foreign students accepted onto engi-neering courses leading to a Master’s Degree.

Agreement for mutual recognition of an engineering degree between the CTI and the CCI dated 21/10/99

Recognition / accreditation of training programs inengineering is a key element to the practice of theengineering profession in any country. Represent-atives of the CTI and the Conseil canadien desingénieurs (CCI) met several times in 1998 and1999 and took part in accreditation / recognitionmissions in France and Canada.

Bologna agreement and students mobility

European Ministers of Education have signed theBologna Agreement. This agreement stipulatesthree levels for the delivery of higher educationdegrees: three years, five years and eight years atthe doctorate level. However, this agreement doesnot stipulate anything about the content of thedegrees and has no specific indication about engi-neering studies.

Usually, European institutions plan to deliver a3-year degree similar in level and specifications tothe bachelor’s degree of the American engineeringschools and after 5 years a degree at the level of the

40

ENQA Workshop Reports

master’s degree in Sciences or in engineering sci-ences.

But many engineering studies in France are or-ganized using a 2–5 years system. After the sec-ondary school, students who have obtained goodmarks in mathematics and physics may enter“Classes préparatoires” for a two years cyclewhere they receive a strong general education inmathematics, physics, chemistry and, eventually,life sciences: roughly 1800 hours of education inbasic sciences useful for engineering studies. Theprogram of these “classes préparatoires” is thesame in the whole France.

The regular duration of studies in such engineer-ing schools is three more years, including roughly2000 hours of courses in specialized sciences,technology, engineering projects, managementand social sciences, foreign languages (in Englisha level equivalent to TOEFL 550 is mandatory),and at least 4 months of engineering practical stayin industry.

Every institution has to adopt the ECTS system.It facilitates entry of 2, 4 or even 6 semesters aftersecondary school and from foreign institutions anduniversities. The Diploma Supplement is under de-velopment.

Gateway to working in industry

The institutions should give priority to four meth-ods of reinforcing the student’s capacity for inno-vation, initiative, entrepreneurial skills and rapidinsertion:

1. Alternation

The study time is divided between training in aninstitute and in-company training. The companytraining is based on a more inductive method,allowing the young trainee to acquire practicalknow-how that will facilitate his/her integrationinto the company culture. Alternation can take anumber of very different forms and applies both toengineering training courses and to apprentice-ships.

2. Project-based learning

Engineers are encouraged to work alternatively in‘hierarchical’ mode and in ‘project’ mode or bothat the same time. Group work around a project isvery close to the professional situation. It showsthe typical development of a project, how it is eval-uated and the typologies: initiation, case study,response to specifications, transversality, bibliog-raphy, etc.

3. Use of new technologies

Another teaching method needs further reinforce-ment: the use of techniques for information andcommunication in teaching on engineering train-ing courses.

4. Professionals in the institute

The decision whether or not to appoint profession-als as teachers is made by the center for recruit-ment of specific permanent posts and the solutionsare worked out individually in each case. Thismakes it possible to transmit professional compe-tence.

CTI criteria

Presenting a summary of the CTI criteria is noteasy, as the standard of training has to be measuredagainst the aims and objectives that each traininginstitute has set itself. The points in common in allthe courses are:

1. General presentation of the establishment2. Curriculum3. Human resources and scientific environment4. Finance, equipment, etc.

Various assessment tools are used. For example itis possible to give a more in-depth assessment ofan establishment and its training programs byapplying more detailed criteria graded from medi-ocre to very good (1 to 5).

Conclusion

In conclusion, the practical efforts of the Commis-sion des titres d’ingénieur have produced someremarkable results.

41

ENQA Workshop Reports

1. Its verdicts on whether or not institutes shouldbe accredited have been accepted and recordedby the responsible ministries without exceptionsince its creation.

2. The CTI has defined orientations and referencesthat are recognized and accepted by all the par-ties concerned.

3. The CTI has the role of an advisor rather thanthat of a censor. It is there to influence and im-prove the results of institutes, to encourage in-novation and attract the best people.

4. The CTI encourages alignment with other na-tional systems of recognition or accreditation tofacilitate mobility for young people and helpthem evolve in their professional life.

5. The CTI experience may well answer to somequestions raised by Viviane Reading: How tocreate the conditions within which universitiescan attain and develop excellence? How to es-tablish closer cooperation between universitiesand enterprises?

2.4 GERMANY: Quality Assurance and Accreditation in Germany

Prof. Dr. Hans-Uwe Erichsen, Chairman of the Accreditation Council, [email protected]

Developments in quality assurance in Germany and at European level

In Germany the Federal States (Länder) areresponsible for the shape and development ofhigher education and research. The responsibilityfor the contents and organisation of studies andexaminations as well as for the quality of highereducation is in principle with the Länder. It hasbeen until recently finally implemented by thelicensing of programmes and definition of therequirements of the exams. According to theHigher Education Framework Act, proposals forstandards of study courses and degrees as well asfor their mutual recognition have been for a longtime made by framework regulations for studies

and examinations (Rahmenprüfungsordnungen),which had to be jointly adopted by the Länder andthe Hochschulrektorenkonferenz (HRK).

The creation of these framework regulations hasproven to be an extraordinarily ponderous proce-dure, often taking many years and producing re-sults which, at the time when they finally wereadopted, had already become inefficient because ofnew developments and therefore proved to becounterproductive, especially with regard to studyprogrammes competing in the international market.

Whereas, quality assurance in teaching in Ger-many was primarily performed through quantita-tive regulations by the state in the way of ex-antecontrol, other countries increasingly pursued qual-ity assurance in teaching on the basis of evaluationresults (ex-post control). Following the interna-tional development and with a growing awarenessof the necessity of quality assurance, a change ofparadigm was claimed in Germany. Based on rec-ommendations of HRK and Wissenschaftsrat,since the mid-1990s evaluation procedures forteaching have been introduced with the goal to in-crease transparency, strengthen institutional re-sponsibility, support higher education institutionsin the introduction of systematic quality-promot-ing measures as well as advancing the profile, im-age and competitiveness of German HE.

Since the beginning of 1998, the HRK runs athree-year national programme to enhance the ex-change of information and experiences in the fieldof quality improvement measures in German HE –the Quality Assurance Project. Moreover, in recentyears evaluation agencies have been established onregional level either by the federal states or by as-sociations of universities. Besides the above men-tioned activities, a lot of departments in many HEinstitutions have started evaluation initiatives us-ing different approaches and different perspec-tives.

As a part of the process initiated by theSorbonne Declaration and advanced by the Bolo-gna Declaration as well as the Prague Communi-qué, it has become clear that the structure of stud-ies and degrees in the European Higher EducationArea in the future will be shaped by “two main cy-

42

ENQA Workshop Reports

cles” and that the scientific community will haveto play an important role in the field of qualitystandard development and assurance. The goalsare to promote international quality standards, toadvance and secure student and graduate mobility,and to improve the employability of graduates onan international labour market.

Introduction of BA/MA study courses and accreditation

The amendments to the Framework Act for HigherEducation (HRG) of 1998 opened Germany’shigher education system for the implementation oftrends and developments at European level. Ger-many’s higher education institutions were giventhe opportunity – initially for a test phase, cur-rently as a normal case – to introduce degreecourses leading to the internationally-recognisedacademic degrees, namely Bachelor and Master.This process especially aims to

• raise the flexibility of study programmes of-fered,

• improve the international compatibility of Ger-man degrees, and thus

• increase student mobility and demand of foreignstudents for study places in Germany.

The introduction of two cycles aimed at restructur-ing and reforming the system of programmes inHE in Germany. Highly adaptable and very flexi-ble contents and time structure should enable HEinstitutions to meet more effectively than in thepast the various and constantly changing demandsof science and education, of professional practiceand of the students. As a consequence, a quickerand more flexible procedure for quality assurancewas necessary. Considering, in addition, the ten-dency to provide the HE institutions with moreautonomy, the system of detailed state control wasreduced and accreditation as a new means of qual-ity assurance was introduced.

Accreditation aims at guaranteeing the nationaland international recognition of (academic) de-grees and, at the same time, at providing higher ed-ucation institutions, students and employers with areliable guide to the quality of study programmes

and degree courses. Accreditation is a flexible toolwith which quality assurance in the fields of stud-ies and teaching can be organised. Beyond thathigher education institutions can use the prepara-tions for accreditation for their own quality assur-ance activities, while the result can be used for thepurpose of international higher education market-ing.

The creation and work of the Akkreditierungsrat (Accreditation Council)

Being aware of the different areas of competenceand responsibility of the state and higher educationinstitutions, the Conference of Länder Ministers ofEducation and Culture decided to stick to a finalresponsibility of the states implemented by licens-ing study programmes. However, focussing onquality aspects they decided together with theHRK to create an accreditation system consistingof the Akkreditierungsrat (Accreditation Council)and agencies being accredited by the Akkredi-tierungsrat and thus being entitled to accredit BA-and MA-programmes. So the German Accredita-tion Council has been established by an agreementbetween HRK and KMK, not by law.

The Akkreditierungsrat is responsible for the es-tablishment of comparable quality standards forBachelor’s and Master’s degree courses in an de-centralised accreditation process. The Akkredi-tierungsrat performs these responsibilities by ac-crediting, coordinating and monitoring the agen-cies.

The 17 Members of the Akkreditierungsrat areHE institutions’ rectors, scientists (among themforeign members), representatives of the states,representatives of the employers and the trade un-ions and students. The Akkreditierungsrat consid-ers student participation in the organisation andpractice of accreditation procedures to be desirableas a means of ensuring that their interests are met,and not least as a means of promoting student ac-ceptance of the new degree courses.

The Akkreditierungsrat had to develop an ac-creditation system. Although experience had beengained abroad with the accreditation of degreecourses, this was “new territory” in Germany. The

43

ENQA Workshop Reports

decentralised system with agencies to be accredit-ed and the Akkreditierungsrat responsible for thecontent and procedures, for the equivalency of re-sults was without precedence and therefore theGerman accreditation system had to be completelydesigned and realized from scratch. In this processthe conflicting interests between the responsibilityof the state, the scientific community’s compe-tence in matters of course content, the profile- andimage-building autonomy of the higher educationinstitutions, and the interests of the labour markethad to be balanced. Being aware of the new ap-proval to a quality assurance system, the Akkredi-tierungsrat has understood itself as a learning sys-tem being prepared and willing to react to mistakesand to cope with new challenges.

Measures for building an accreditation system

The present accreditation system has to guaranteeminimum standards of quality of programmesleading to a BA- or MA-degree. Up to now, thesystem does not allow to certify a special top qual-ity. In order to be able to meet its responsibilitiesof providing comprehensive quality assurance andcontrol, the Akkreditierungsrat adopted principlesand minimum standards which agencies had tomeet.

So the Akkreditierungsrat has passed some reg-ulations obligatory for the organisation of theagencies: they must be independent from the state,HE institutions, associations of faculties(Fakultätentage) and disciplines (Fachgesellschaf-ten), professional associations and business, theyshall not be profit-oriented and they must performaccreditation for all types of HE institutions of thestates. They have to be organised as a legal entity,having, according to the form of organisation cho-sen, a body which has the final decision on the ap-plication and which has to be composed corre-sponding to the membership in the Akkreditierung-srat.

The accreditation procedure aims at ensuringequivalency, guaranteeing quality, creating trans-parency and also encouraging and facilitating di-versity. Only if the agencies observe a frame ofreference, i.e. agreed criteria, standards and proce-

dures, when they accredit degree courses, accredi-tation results can be regarded to have met the con-dition of equivalency. One main task for theAkkreditierungsrat therefore was to develop crite-ria to be applied when agencies accredit degreecourses. The intention of allowing higher educa-tion institutions as much freedom as possible instructuring their courses, without, however, jeop-ardising the comparability of future study pro-grammes, led to relatively general criteria. In con-trast to the somewhat rigid quantitative standardsand specifications contained in the framework ex-amination regulations, the criteria now provide aflexible framework for the review of degree cours-es. Since neither the Bachelor’s degree nor theMaster’s degree are finally defined, standards con-cerning level and work load had to be developed.In the test phase for the new degree courses, whichaims to promote innovation, it seemed acceptableto have only a few criteria serving as a rough ori-entation.

The accreditation procedure prescribed by theAccreditation Council (Akkreditierungsrat) startswith an application. The final decision of the ac-creditation commission of the agency has to beprepared by peers. The peers have to be selectedfollowing certain procedural and quality aspects.

The Akkreditierungsrat developed special re-quirements for the accreditation application. Thesespecifications mainly cover:

• “reasons for the degree course” (e.g. missionstatement, goals and aims of the degree course),

• the planned “degree course structure and re-quirements in terms of content and specialisa-tion“ (e.g. organisation, structure and content ofthe programme, professional qualification ofgraduates on the basis of a consistent and coher-ent programme, assessment of the foreseeabledevelopments in potential field of labour mar-ket),

• “human, financial and infrastructural resources“(e.g. qualification of the staff, funding of theprogramme etc.),

• “quality assurance measures“ (e.g. data on com-pletion rates, student satisfaction etc.) and

44

ENQA Workshop Reports

• “study-related cooperation“ (esp. concerninginternational programmes, e.g. cooperation withinstitutions abroad, students from other coun-tries etc.).

The introduction of the new Bachelor’s and Mas-ter’s degree courses has at the end supported andencouraged growth in the introduction of innova-tive degree courses by higher education institu-tions and faculties.

After giving careful consideration to the factthat evaluation and accreditation serve differinggoals, the Akkreditierungsrat tried to ensure thatthe various procedures are separated and that eval-uation and accreditation are carried out in separateconsultative and decision-making committees andon the basis of separate procedures. However, theAkkreditierungsrat supported the view that recentevaluation results must be considered in accredita-tion decision-making.

Accreditation and coordination of agencies

In the meantime, seven5 German agencies havebeen accredited and are thus entitled to award theSiegel des Akkreditierungsrates (Quality Certifi-cate of the Akkreditierungsrat) to the Bachelor’sand Master’s degree courses of state Higher Edu-cations Institutions. Other agencies, also fromabroad, have announced their intention to submitapplications.

The Akkreditierungsrat guarantees equivalencyand quality within diversity by defining minimumstandards, certain procedures and organisationalstructures, by coordinating the procedures prac-tised by the accreditation agencies and by under-taking follow-up monitoring measures. In particu-lar, it monitors the observance of minimum stand-ards of quality and procedure, the implementationof conditions imposed on the agencies as well asthe execution of other resolutions adopted by theAkkreditierungsrat. The agencies have to reportevery accreditation of degree courses together withthe review report to the Akkreditierungsrat. Be-yond this, the agencies have to submit an annualreport as part of their accountability obligation.

Moreover, members of the Akkreditierungsratmay, in agreement with the agencies, attend, asguests, sessions of the agency’s decision-makingcommittee or peer-review team sessions. In orderto guarantee transparency, the Akkreditierungsrathas resolved to make the accreditation decisionspublic at the end of the accreditation procedure.You can find information on accreditations ofstudy courses (ca. 3806) on our website.

Agreement and cooperation in the international field

The Akkreditierungsrat introduces and representsGerman views in international discussions on (aca-demic) degrees, transparency, quality and stand-ards in higher education. Contribution to informa-tion exchange and to cooperation in Europe aim atthe acceptance of the Quality Certificate of theAkkreditierungsrat in Europe and abroad. Agree-ments on mutual recognition must be reached, inorder to avoid multiple accreditations. The goal isto ensure that accreditation achieved abroadshould to be recognised in Germany and viceversa.

In order to reach this aim, the Akkreditierung-srat maintains contacts with international accredi-tation institutions and organisations which per-form comparable functions and responsibilities.The Akkreditierungsrat is a member of the Interna-tional Network for Quality Assurance in HigherEducation (INQAAHE), the European Networkfor Quality Assurance in Higher Education (EN-QA), the European Consortium for Accreditation(ECA) and the D-A-CH network, the network ofaccreditation institutions of Germany, Austria andSwitzerland. These memberships promote ex-change with agencies abroad and help make theQuality Certificate of the Akkreditierungsratknown at international level.

The option of accreditation which has becomepossible as a result of the introduction of an ac-creditation system has provoked an intensive dis-cussion in the federal states (Länder) and highereducation institutions on restructuring curricula,contents and on the quality assurance of new de-

5. Two agencies have merged so that there at the moment(November 2003) six agencies are operating. 6. As of November 2003.

45

ENQA Workshop Reports

gree courses. Higher education institutions feel en-couraged to implement new and innovative ideas.

The system established by the Akkreditierung-srat has to be stabilised in the future. The systemof accreditation and its results must be made betterknown abroad and it must be further developedtaking into account international developments.

The test phase has been finished by a new stat-ute of the accreditation system in Germany whichcame into power at the 1st of January 2003. Thereare some modifications especially concerning thecompetences of the accreditation system. It will nolonger be restricted to the accreditation of newlyestablished BA- and MA-courses, but the range ofcompetence is extended to newly set up Diploma-and Magister-courses and to Diplom and Magistercourses which shall be fundamentally changed, insubjects for which there are no framework exami-nation regulations or for which the existing frame-work regulations are out of date.

2.5 HUNGARY: Accreditation Models in Higher Education: Experiences and Perspectives

Christina Rozsnyai, Programme Officer, Hungar-ian Accreditation Committee, [email protected]

Introduction

The paper is a summary of two presentations, thefirst given at the session on the “Scope of accredi-tation“, the second in “Working methods“. Bothdealt with the Hungarian experience in qualityassurance in higher education. In addition, the firstpresentation briefly looked also into the practice ofaccreditation in Central and Eastern Europe, whichis described in conclusion.

The Hungarian Accreditation Committee

The Hungarian Accreditation Committee (HAC)was set up by the Higher Education act in 1993,

mandating accreditation of all higher educationinstitutions and all their programmes every eightyears. The first cycle (at the time of 89 institutions)took place between 1995 and 2001. The HAC’sconcept of institutional accreditation was based onthe premise that the output of higher educationinstitutions was a diploma or degree, and the con-tent behind the degree was the study programme,therefore it must be the object of evaluation. Theinstitution was seen as the environment contribut-ing to the quality of study programmes. The insti-tutional level, in contrast, was not of equal signifi-cance because in the social-historical context inwhich Hungary found itself after regime change inthe early 1990s, there was little experience in insti-tutional management, and institutional leaderswere selected based on academic merit. Linked tothat, legally declared institutional autonomy wasin fact limited, with numerous aspects of highereducation legislated and severe financial restric-tions imposed both by legislation and the amountof money available and allocated to higher educa-tion. Thirdly, there was no internal quality assur-ance in place at the time.

The accreditation decision by HAC pertained,therefore, to a whole institution, all its faculties,and all its study programmes. Roughly one third ofthe programmes were given “conditional“ accred-itation, with defined conditions to be met by a setdate, reviewed in a monitoring procedure. Somesmall, new colleges were also given short-term,conditional accreditation and no institution wasclosed. There was small number of new institu-tions requesting preliminary accreditation that hadto resubmit their application before being grantedtheir request.

In the upcoming cycle of institutional accredita-tion, beginning in autumn 2004, greater emphasiswill be given to the institutional level. Internalquality assurance is in place at all higher educationinstitutions, who send their annual reports, review-ing changes in their institution and programmes aswell as quality concerns, to the HAC. The reportswill constitute the building blocks for accredita-tion. A selection of programmes will be reviewedin depth. Whereas in the first cycle, only the ac-

46

ENQA Workshop Reports

creditation decision and a brief explanation for it,but covering the institutional level as well as theprogrammes, was published, accreditation reportswill now be published in full. The HAC has al-ready launched a pilot procedure in which it eval-uates a specific discipline across the board, where-by the same visiting team reviews the study pro-grammes in the given discipline at all institutionsin the country within a limited time-frame. The pi-lot phase, still running at the time of this writing inearly 2004, covers the disciplines of psychologyand history, and no decision has yet been takenconcerning the feasibility of the approach in the fu-ture.

The HAC has 30 full members, who are delegat-ed by higher education institutions (the HungarianRector’s Conference, the Conference of CollegeDirectors, and the Conference of Art UniversityRectors); by research institutes (of the HungarianAcademy of Sciences); and by professional organ-isations (chambers, unions). There is also one non-voting student member, as called for by the highereducation act of 1993. In addition, several non-voting members are invited on a permanent basisto fill in for major disciplines not covered by thedelegated members. This is necessary, since theHAC works in a multiple-level decision-makingstructure both in institutional accreditation andseparate programme accreditation procedures. Thelatter involves the preliminary accreditation ofnew programmes on the national level (initiated byinstitutions but issued as national qualification re-quirements in the form of government decrees);the preliminary accreditation of new programmeslaunched by institutions (based on the nationalqualification requirement for the given study pro-gramme); and the preliminary accreditation ofdoctoral schools. Moreover, as noted, programmeaccreditation is also part of institutional, that is expost, accreditation every eight years.

The internal procedure for conducting institu-tional accreditation (which involves visits by apeer review team sent to each faculty and based onthe institution’s self-evaluation report) is as fol-lows. The members of the HAC plenum headstanding expert commissions for main disciplines

or discipline groups. As expert commission chairsthey recommend the leaders of the review teams.The review team leaders in turn recommend themembers of the team, which may include non-aca-demics. The team is approved by the institution tobe visited, and approved by the HAC plenum. Inthe following cycle of institutional accreditation,students will participate in visiting teams. Anotherdifference between the first and second cycles isthat in the former the accreditation decision wasmade on a grading scale of Excellent, Strong, Ad-equate, and Not Adequate (with excellent beingmeasured against the international standard),which will be discontinued in the new cycle, leav-ing only a yes/no decision. In both cycles there wasand continues to be Conditional Accreditation(technically a yes decision), either if there were notyet any graduates in the evaluated programme or ifweaknesses called for a monitoring evaluation,whereby set conditions must be met by a givendeadline. The visiting team produces an evaluationreport that is discussed by an ad hoc commission,made up of HAC members representing the disci-plines evaluated, with a final accreditation reportpassed as a resolution by the HAC plenum. Prior tothe final vote, the institution is given the report forcomments. The final report is published.

The selection of evaluators for programme ac-creditation, which involves evaluation based on awritten application, proceeds as follows. The chairof the relevant expert commission for the givendiscipline (usually a plenum member) recom-mends two external evaluators, usually but not al-ways academics. A third evaluator may be calledupon if the evaluation is not unambiguous. TheHAC has a pool of over 500 peers. The expertcommission discusses the evaluations and pre-pares its recommendation for the plenum, whichpasses the final decision on granting preliminaryaccreditation to a new programme in the form of aresolution.

With both institutional and programme accredi-tation, a HAC decision is an “opinion“ given to theMinister of Education, who issues the final deci-sion on accreditation. By law, the Minister mustpublish his or her reasons for passing a decision

47

ENQA Workshop Reports

that is contrary to the HAC’s opinion. Institutionshave the right to appeal the HAC’s decisions basedon legal grounds. The frames of reference for theHAC’s decisions are the higher education act; thegovernment decree on the HAC that details thedelegation of HAC members and the tasks; theHAC’s By-Laws, which include procedures of op-eration and tasks of the committees; the HAC’sAccreditation Requirements; the HAC’s StrategicPlan; and its Code of Ethics.

All higher education institutions which appliedfor accreditation have been accredited (about halffor the full eight-year term), and about 70% of theprogrammes were accredited for the eight-yearterm, while less than 1% were closed. Almost allprivate higher education institutions applying foraccreditation were accredited, though some had tore-submit their application. There are now 11 pri-vate HEIs in Hungary.

Accreditation in Central and Eastern Europe

Quality assurance in higher education in CEEcountries began with the main aim to protect stake-holders by insuring the quality of higher educationin the respective countries. It took the form ofaccreditation in almost all CEE countries from thestart and is now being conducted in all countries.The reasons for this choice have been discussed indetail in the literature, but mainly had to do withthe fact that higher education policy-makers, inconjunction with established academics, saw aform of control necessary at the time of regimechange, whereby institutions were granted a cer-tain degree of autonomy in exchange for allowingexternal control of the quality of the education theyproduced. In the given social-historical context theaccreditation structure may have appeared as rigidand, indeed, the practice varies in the differentcountries. Other reasons for introducing accredita-tion in CEE countries was to protect stakeholders;to define quality standards and levels; to assurecomparability of study programme content andlevel with those in Western Europe; and in somecountries, most notably Romania and Bulgaria, tocontrol the quality of education at proliferating pri-

vate institutions. A survey showed that all qualityassurance agencies professed an orientationtoward helping higher education institutions toimprove the quality of their education.

All CEE countries have national quality assur-ance agencies, although Poland until recently hadonly commissions set up with the voluntary co-op-eration of higher education institutions of variousprofiles. In recent years, as the new social struc-tures are taking root, a development toward a moreflexible implementation of quality assurance and arelaxation in the legislation can be witnessed.Higher education laws are being revised or newones passed in several countries.

The Central and Eastern European Network ofQuality Assurance Agencies in Higher Educationwas formally established on October 19, 2002 inVienna as a non-governmental and non-profit or-ganisation. The CEE Network has 18 membersfrom 16 countries. The contribution of CEE agen-cies to the dialogue on quality assurance in Europeis to define educational and quality assurance strat-egies in each country; to co-operate among eachother to define the needs and expectations for high-er education and quality assurance; to channeltheir opinions to other European players in qualityassurance; to participate in European projects in apro-active way and to initiate own projects in orderto arrive and mutually acceptable and comparablestandards and methodologies in quality assurance.

2.6 ITALY: From Authorization to Accreditation – A Difficult Path for the University System in a Changing Society

Alessandro Figà Talamanca, CNVSU board and The University of Rome “La Sapienza“, email: [email protected]

Why did the problem of “accreditation“ seem tohave surfaced in Europe only in recent years? Ishall not attempt to give a general answer to this

48

ENQA Workshop Reports

question. I will rather try to explain why it was notan important issue in Italy until very recently andwhy accreditation is a difficult problem, in view ofthe many changes that the Italian University sys-tem is currently undergoing. The remarks pre-sented here could also apply to other Europeancountries, especially of continental Europe, andalso to countries where the accreditation of univer-sities has not yet come to full public attention.

We are in Rome, which is one of the oldest citiesin Europe, and whose oldest university celebratesits 700th anniversary this year. I should be ex-cused, therefore, if I go back to the medieval be-ginning of the Italian University system. After abrief period of spontaneous generation andgrowth, Italian universities, not unlike universitiesof other European countries, were established byan explicit act of a religious or civil authority,which authorised their activity conferring, in gen-eral, special privileges to the faculty and to the stu-dents.

Probably the first university explicitly estab-lished by a ruler, to fulfil a “political” mission, wasthe University of Naples, which has now assumedthe name of its founder “Frederick II”. It was in-deed created in the first half of the 13th century, byFrederick II, Roman (which, at the time, meant“German”) Emperor, and King of Naples. TheUniversity of Naples was established to contrastthe political influence of the Law School of theUniversity of Bologna, a city which, like most Ital-ian cities of the North at that time, sided with thePope, in the century long struggle for world su-premacy between the Papacy and the Empire. Ac-cordingly, Frederick II forbade his subjects toteach or to study at the University of Bologna.Similarly the Pope established the University ofRome, exactly 700 years ago in 1303.

At the end of the nineteenth century, when theKingdom of Italy succeeded to the many sovereignstates of the peninsula, the 22 universities operat-ing in the Italian towns passed under control of thecentral Government. Government control meantthat the charter approved by the Ministry of Edu-cation, explicitly listed the schools or faculties,which were authorised and the courses of instruc-

tions and degrees which these schools could offer.Even the selection of the professors hired to fill thechairs was organised on a national basis, movingaway, in this respect, from the model of Germanuniversities, at the time the paradigm of a modernuniversity system.

“Authorisation” automatically implies “accred-itation” in a system which is rigidly controlled bya central authority. Under these conditions, thereshould be no need for a system of accreditation.Whatever is authorised is automatically accreditedand it is the Government who is responsible forproviding the necessary staff and facilities to theinstitutions for the courses of instruction allowedin their charters.

It is natural to ask why we talk of “accredita-tion” under these conditions, which are not farfrom the conditions under which operate many Eu-ropean universities. Is it because we blindly followNorth American fashions and North American ter-minology, ignoring the fact that we operate in avery different context? Is it because small groupsof bureaucrats, firmly entrenched in the QualityAssurance Agencies of each country, want to con-solidate and extend their power over the universitysystem?

There is, perhaps, a grain of truth in these expla-nations. But it is also true that an apparently minorchange occurred in the relationship between thecentral authority and the university system in Italy,which has become one of the reasons to introduce“accreditation”. The change can be synthesised asfollows: in order to gain a better control over theuniversity system, universities are given free reinon how to spend the money they receive from theGovernment, but the Government reserves theright to measure the level of financing against re-sults achieved. Of course the Government fixes thepriorities in terms of results.

This change came about when it was realisedthat the system of rigid control through authorisa-tion, detailed regulation, and minute indications onhow the money should be spent, made it impossi-ble to control the growth and the global expendi-ture of the system. The growth of the universitysystem through the seventies and the eighties in It-

49

ENQA Workshop Reports

aly was characterised by intensive lobbying by lo-cal authorities and politicians, and by academicgroups, to obtain, even in absence of the necessaryfacilities, one or more “authorisations” to initiatenew courses of instruction, or to give life to newuniversities. Very little attention was given to theproblem of adequately financing the new institu-tions or the new courses of instruction. It was un-derstood that the government would have to pickup the bill sooner or later.

For a while, until the late eighties, it was thoughtthat the growth of the university system could stillbe controlled through “three year plans”, approvedby the Ministry on the advice of ParliamentaryCommittees. But the results were not particularlybrilliant.

Finally, in the middle nineties it was decidedthat in order to control the expenditures of the uni-versity system it was more convenient to give theuniversities free rein in the allocation of resourcesobtained from the central government. Rather thanfooting the bill for each “authorised” activity, theGovernment would provide a global funding,measured on the basis of a “formula”, based on in-dicators of results. In other words, the allocation ofresources to universities was measured against ob-jective results of university activity, rather thanabstract needs. At the same time, universities werefree to choose their own strategy in achieving theirgoals.

As a result, the universities are induced to com-pete for more money from the government and, asa consequence, for better performance. We haveintroduced what may be called a “simulation” ofmarket conditions.

But as every economist knows, competitionamong service providers does not always producepositive effects. Any funding of university institu-tions which is based on “results” is heavily de-pendent on the number of students, or at least thenumber of graduates. The universities are thus in-duced to increase the number of students by offer-ing new courses of study on topics attractive to stu-dents. This may lead to a form of deceptive public-ity, which requires a corrective intervention on thepart of the central authority.

In general, the type of “quasi-market”, which isinduced by the new system of financing the univer-sities, requires that the offer of instruction on thepart of the universities is “accredited”, because ofthe “asymmetry” of information between users andproviders of the instructional services. A centralauthority must define rules on the basis of whichthe providers of services are admitted to the mar-ket. Accreditation, a concept that only a few yearsago could be ignored by Italian universities, andwhich seemed to apply only to the American sys-tem of higher education, has become an issue in It-aly, and indeed in most European countries.

At the beginning we may have thought that ac-creditation should only apply to newcomers to thesystem, or perhaps only to universities which arenot totally sponsored by the State. But as timepasses these limitations appear less and less rea-sonable. Every institution is a “newcomer” when itstarts a new course of instruction, or shifts its re-sources from one area to another. Indeed, the mainpractical difference between accreditation and au-thorisation is that the former is never permanent,but it is subject to periodic reviews. We must bringourselves to accept that accreditation concerns thewhole university system.

What do we mean by accreditation of a univer-sity? We must bear in mind that accreditation im-plies setting of “standards” for an activity, whichtraditionally prided itself in being above “stand-ards”. It is not too difficult to set minimal require-ments in terms of number of permanent faculty, li-braries, laboratories and other facilities. Muchmore difficult is to indicate reasonable standardsof performance for a university. Let me state in thiscontext just two of the many problems which couldbe raised.

One serious problem is the definition of the lev-el of instruction suitable for a university education.Should it be defined independently of the level ofcompetence and prior education of entering stu-dents? Or should it be calibrated on the actual levelof the student body, no matter how low?

In Italy, as in most European countries, univer-sity education is no longer reserved to a élite. It isexpected that the university system address itself

50

ENQA Workshop Reports

to a high percentage (at least 30%) of the popula-tion of young people. Under these circumstancesthe level of instruction must adapt to many differ-ent needs, expectations, and prior education of adiversified student body. It seems reasonable notto deny accreditation to an institution which takesupon itself the task of teaching students who do notmeet the highest standards in terms of prior educa-tion. But if accreditation is supposed to have anyrelevance outside the university system, it shouldsay something about the level of competence,which is expected of the graduates. We are regis-tering here a conflict between the duties of the uni-versity system to address itself to a larger and larg-er percentage of the student population and theneed to be accountable to the public and the pro-spective employers for the level of competence ofthe graduates.

A second problem, which is connected with theformer, is the question of the role played by the sci-entific research of the faculty in the definition of auniversity. Is it still reasonable to expect that alluniversity faculties be actively engaged in re-search? If so, does this mean that the standards foraccreditation should include an evaluation of theresearch of the faculty? How far can a country af-ford a university system addressing itself to 40-50% of the population of young people, where allfaculty members are allowed time and resourcesfor creative research?

On the other hand, a university is traditionallythe place where scientific research combines itselfwith teaching to provide an intellectual environ-ment that should foster creative thinking. Univer-sity teaching has been for the last two centuries themost important vehicle for the “transfer” of scien-tific and technical innovation to society and theproductive world. Under these circumstances, auniversity without research may be considered acontradiction in terms. The solution adopted bysome countries in the sixties, which was to createinstitutions of higher education which are not fullyuniversities is now being abandoned, as the exam-ples of England and Sweden show. Do we reallyhave other solutions?

2.7 An Approach to Accreditation: The Path of the Italian Higher Education

Carlo Calandra Buonaura, CNVSU Board and University of Modena and Reggio Emilia, [email protected] Di Nauta, CNVSU Technical Secretari-at, [email protected]

To understand the subject of accreditation in Italyit is important to focus the attention on some initi-atives which have been recently taken by the Min-istero dell’Istruzione, dell’Università e dellaRicerca (MIUR) under the advice of the ComitatoNazionale per la Valutazione del Sistema Universi-tario (CNVSU) that have a strong impact on thewhole university system. And it is also importantfor CNVSU to have indications, criticisms andideas concerning these initiatives as they are a partof a strategy that aims at the gradual introduction ofaccreditation standards in the Italian university.

The basic idea is just to have gradual develop-ments in the university system related to the use ofgeneral quality standards. In fact, graduality is amust in the present period that is characterized bytwo important changes: an increasing decentralisa-tion of functions and responsibilities, and a com-plete modification of the university degree system.

Universities are going through a transition froma highly regulated to a more autonomous system.Faculties are facing with the difficulties of deter-mining their objectives and qualifying their pro-grammes to compete in a relatively free market ofhigh education providers. This process does not oc-cur only in Italy, is a global process started at Euro-pean level. The implication is that there is a sort ofincrease of responsiveness of the university to soci-ety demands, and this concerns all the activities ofuniversity system: education, research, expertise.

The Italian education system enumerates 77universities, 63 state universities and 14 non-stateones. Universities are the main institutions for thedelivery of degrees at high education level, and ofcourse for performing scientific research. Financ-ing, funding and evaluating universities, is handled

51

ENQA Workshop Reports

at a ministerial level. The student population is inthe order of 1.6 million students, but the number ofregular students is smaller, about fifty per cent.One of the reasons is that not all students are fulltime students, many of them are part time, and thisis a somewhat new reality. Indeed one of the diffi-culties that universities may have in offering edu-cational programmes is just to take into accountthis diversification of the student population.

A second change has been then introduced withthe Bologna process that caused a real revolutionin the higher education system in Italy with the in-troduction of the two-level educational system andof the educational credits. Before the Bolognaprocess, at the end of high school education, stu-dents could choose between a diploma course (athree-year course) and the traditional laureacourse (from four to six years, depending on thesubject). The diploma course has always been cho-sen by a small percentage of students, most pre-ferred to get the more qualified, more common,more traditional laurea course, considering alsothat after laurea graduation there was the possibil-ity, for a restricted percentage, to enter the PhDsystem. It was a rather rigid system, especially be-cause it was ruled at a national level by a detailedregulation of the educational programmes. Eventhe name of the courses and the context of the sin-gle courses were ruled by law. The idea was to of-fer the same standard all over the country. Sincethe programmes were supposed to be the sameeverywhere, accreditation was considered not nec-essary. Of course, there were significant differenc-es in quality among universities, but the principlewas that the programme was the same everywhereas, once it was approved by the Ministry, it was au-tomatically “accredited”.

The new system is completely different, and it isvery difficult to know the kind of graduates it isproducing, as it started three years ago, and thefirst students are graduating just now. It is organ-ised in two levels, a first level, with entering selec-tion in some courses, and a second level, laureaspecialistica, which is supposed to be culturallymore professional in character. One year courseson specific professional subjects (in Italy they are

called Masters) can be taken after both degrees,getting a first level Master or a second level one.Of course there is also the PhD school with a veryrestricted selection.

According to the rules of the new system, pro-grammes on offer are classified on the basis ofclasses (42 for the first three year courses and 104for the second level). The institutional mission ofthe educational programme of each class is broad-ly defined by law. Within each class there is a min-imum number of credits for specific disciplinesthat are determined by law.

Universities, the providers, are expected to com-plete the programmes until the proper numbers ofcredits is achieved, which implies that roughly 60-70% of the programme is defined by universities,independently of any indication from the law. Evenafter the reform, the existing universities had main-tained the rights of awarding degrees at all levels,including PhD programmes; in other words thereare no restrictions for universities, and they can of-fer all the three levels of degree.

In principle each university is responsible forthe quality of the educational provisions, but theMinistry set some basic quality standards, whichhigher education programmes are supposed tomeet. As universities are autonomous, nothing pre-vents them from offering programmes that do notmeet the standards, except the fact that they do notget funding. In principle, if they want to offer lowquality courses, they could. This is the main prob-lem; the increased freedom given to universitiesraises the problem of their accountability.

The introduction of the new system requires aperiod of adjustment, and it is necessary to adaptinstitutions and their high education programmesto the new reality. That is why it would not be sowise at the moment to determine rigid and detailedaccreditation standards; it seems more convenientto use simple quality standards that can be helpfulto guide the universities and the stakeholders in thetransition. There is not an enough experience ofthe new system to allow a systematic accreditationof universities and courses.

One of the problems, definitively to be solvedvery soon, is whether to go in the direction of insti-

52

ENQA Workshop Reports

tutional or programmes accreditation consideringthat most universities have a long standing tradi-tion, and it would be expensive, time-consuming,and probably not convenient now, to start an insti-tutional accreditation process. Of course the situa-tion is different for new universities, for which in-stitutional accreditation can be performed to mon-itor their achievements, and check whether theirstandards in terms of educational offering, in termsof realisation and infrastructures, in terms of serv-ice to the students, are acceptable. This is indeedwhat CNVSU is doing at the moment, followingthe starting-up of new universities with site visits,checking their activities, while, in all other cases,that is to say long standing old universities, theMinistry decided to start a form of accreditation ofthe programmes on offer.

At the moment, most universities will completethe first cycle of the first level degree in the currentacademic year, and this means that the first stu-dents are going to get their three-year degree,while the first cycle for the second level degreestarts just with the academic year 2003/04.

In this background there are at least two basicproblems to be considered.The first one is the in-formation on the educational programmes, as uni-versities are free to determine their educational of-fer, there is a large variety of proposals of pro-grammes, which differ in the design of the courseand curriculum. How can students and stakehold-ers be guided towards the proper choice? The sec-ond problem is how to establish some accredita-tion procedures for these programmes, based onfundamental quality standards, to be improvedwhile the transition goes on?

To better understand the situation it is necessaryto spend few words on the Italian evaluation sys-tem. It has been introduced with the purpose ofdriving the system toward a better performance. Itis based on a network with four main actors: Min-istry, Universities, CNVSU, and the Internal Eval-uation Units (IEU). CNVSU is an independentboard of experts appointed by the Minister, withthe task of advicing and consulting to the Minister,as all the decisions concerning the introduction ofquality standards are taken by the Ministry. The

IEUs are self evaluation university groups of ex-perts appointed by the Rector of each university.There is a systematic link between the CNVSUand IEUs: IEUs collect experience and data, whileCNVSU provides methodology, feedback, indica-tions, and suggestions. This is a kind of virtuousloop, which allows to systematically improve data,procedures and so on.

Getting back to the problem of information, oneof the purposes of accreditation is to guarantee thatpotential students can attend programmes whichpass through some processes of evaluation that en-sures acceptable quality standards. In principlestudents should be given equivalent good qualityeducation regardless of their choice of the highereducation institution. They should be able to com-pare programmes of different universities, in orderto choose the courses provided with good qualifi-cations. Such a choice should be based on real andupdated information about the quality of the cours-es rather than on perceptions. This is the reasonwhy MIUR decided to create the database of pro-grammes on offer (Banca Dati dell’Offerta Form-ativa, BOFF). This database has been establishedwith the purpose of providing students with quali-fied and comparable information on the educationprogrammes for all Italian universities. Studentscan get information by simply visiting the BOFFweb site. Every year, each provider is expected topublicise education programmes on offer, as wellas the curriculum design and the course organisa-tion, through the BOFF database. In the same da-tabase the student can find information about thematching of minimum quality standards for ed-ucational programmes, set by the Ministry underthe advice of the CNVSU.

Quality standards provide a quality thresholdfor all educational programmes. They are based onthe principle of sustainable high education pro-grammes. According to this principle a universityis expected to offer programmes with a propernumber of teaching professors, a proper size of thestudent class, a proper qualification and compe-tence of the teaching staff, the availability of thenecessary infrastructures (libraries, classrooms,teaching laboratories and so on). This could seem

53

ENQA Workshop Reports

a very obvious thing, but it is not always so obvi-ous as, according to their complete autonomy, uni-versities have to compete. Each university tends toattract more students and, to gain competitive ad-vantage, it could deliver programmes without theproper quality.

This is a very basic and very rough form of ac-creditation, an accreditation in the sense that theoutcome is a yes or a no, a match or non-match ofstandards. The threshold is not fixed once for all,but can be raised by further qualifications of allstandards. Improving the standards is expected toinduce a better quality of the programmes. Thisprocedure does not aim at excellence, but has thepurpose to make stakeholders confident with thoseeducational programmes on offer that reach a cer-tain level of quality.

It is interesting to look at the relation betweenthis sort of accreditation and funding. As a result ofthe accreditation process, university programmesacquire a label, indicated in the (BOFF) database,so that stakeholders are informed of the achieve-ment of the minimum quality standards, which al-lows them to be eligible for Ministry funding.

For this reason it is essential that the assessmentbe based on predefined standards and objectivedata, to ensure the transparency of the procedure.At the present stage, everything is based mainlyon quantitative indicators, collected and treated inthe proper way, to reduce the number of possibleerrors: data of programmes are linked with dataconcerning professors and students. An importantrole is played by IEUs, which are responsible forthose aspects that cannot easily be expressed bynumerical indicators. Just to give some results, inthe academic year 2001/02 there were 2.650 firstlevel course programmes set up for 190 faculties.Approximately 75% of the faculties compliedwith the minimum quality standard qualificationsin all the programmes, 25% of faculties did notcomply, that means that 25% of the faculties hadsome programmes which were not acceptable.This was just an experimental approach startingwith a rough kind of quality standard, like for ex-ample the number of professors, not their qualifi-cation.

As expected, the most difficult situation was inthe most crowded faculties like Economics, Law,and Humanities in general. The percentage of pro-grammes in these faculties which did not fit all thequality standards, was of the order of 20–35%. Notonly crowded faculties, also recently establisheduniversities, had insufficient infrastructures, andsometimes also an inadequate teaching staff.

In the second year (academic year 2002–2003)CNVSU started improving the threshold, includ-ing a criterion of the proper qualification of theteaching staff involved in the programmes. Thisimplied a check on the disciplinary specialisationand personal research area of professors, linkingthis to the setting up of the course programmes. Ofcourse, the target was not that 100% of the profes-sors had to be in the right scientific area, as somebasic courses can be taught by professors of differ-ent specialisations or from external experts, but atleast 40–50% of the professors had to have theright qualifications. The results obtained by intro-ducing this threshold are somewhat different: insome cases the situation has gotten worse, since in-cluding more qualified standards caused more fac-ulties not to meet the quality threshols. In othercases it turned out to be better, because some pro-grammes of the previous year, which did not meetthe standards, had been cancelled in the meanwhileby the universities.

The introduction of more severe quality stand-ards has led to a reduction of the accredited pro-grammes and to serious problems in recently es-tablished universities. In other cases there has beena significant reduction of the programmes belowthe standards, largely as a consequence of the de-creased number of programmes on offer by someuniversities. In other words, after the first experi-ment, some universities realised that they were ac-tually offering more than they could afford, sosome programmes had to be cancelled. This is notsurprising in an experimental transition, as maybeuniversities realised to have designed not so an ap-pealing offer, or that the evaluation system of qual-ity did not allow to consider the offer as a valuableone. This is the present stage of accreditation forthe first level degree.

54

ENQA Workshop Reports

The second level degree started in the academicyear 2003/04 with three possible requirements in-dicated by CNVSU: (i) proper publicisation of theaccess, (ii) attractiveness and (iii) scholar scientif-ic production. The publicisation of the second lev-el programmes is very important, because enteringthe second level is a different step in education.The offer should specify the conditions to be reg-istered, the number of proper credits, the nature ofthe accepted credits, the nature of the accepted de-gree, and how to acquire missing credits. Attrac-tiveness is a very important issue, as in Italy stu-dents generally show a low mobility. They usuallychoose the nearest university rather than the onewhich maybe the best for them. This maybe ac-ceptable for the first level degree, but not for thesecond level, that is supposed to be highly profes-sional. The idea is to have some standards of at-tractiveness, and if a university is not able to attractstudents from other regions or from abroad, maybeit could be indicated as not meeting the essentialquality standards.

All the issues, we have been dealing with so far,concern an ex ante accreditation, something whichrefers to the offer and not to the outcome. The re-form of the educational system has established thegeneral purposes and achievements of any class ofprogrammes, and this means that the next step willbe to look at the outcome, to have some assessmentof the results of the educational process. There areseveral issues to be analysed, like the characteris-tics of the educational process, the professionalachievements, the rating of the graduates on thejob market, and so on.

Some important steps have been taken by CN-VSU in order to have the necessary tools for theanalysis of the results of the educational pro-grammes. An information system has been set upthat provides every year data on the student statusin each programme (failure rates, success rates,satisfaction of students, …). It is still a quantitativeelement, not qualitative, but of fundamental im-portance.

CNVSU introduced also a feedback question-naire to be filled in by the students at the end of theprogramme, with a minimum set of questions. As

a result now nearly 60% of the teaching courses inthe programmes have a feedback by students.

Another step is the introduction of a question-naire for the students that get the degree, beforethey leave the university. This allows them to givea global judgement of the university experience.

Moreover the Ministry has supported the reali-zation of an important database, Alma Laurea,which allows to get information on the profession-al status of graduates five years after the end oftheir university programme. This should be a goodstep towards an analysis of what comes out fromour high education system.

Italy is in its early stages of accreditation, in itsinfancy, and this could be common to other Euro-pean countries, but some important results havebeen achieved in the direction of sustainability ofhigher education programmes, and CNVSU isworking strongly to prepare tools for a wider ac-creditation.

2.8 THE NETHERLANDS: Accreditation in the Netherlands

Karl Dittrich, Vice-President, The Netherlands Accreditation Organisation, [email protected]

1. Accreditation in the Netherlands has beenintroduced in the reform of Dutch H.E. towards thebachelor-master structure. Accreditation is seen asthe independent proof that a certain quality-levelhas been reached by a programme.

During the discussions between minister and H.E.-institutions, and minister and parliament, fourgoals for accreditation have been mentioned:

a. Accountability: politics, public opinion, the“taxpayer” ask from H.E.-institutions to be ac-countable for what they do with the money thegovernment funds them with. Accreditation isone of the methods to show that quality has beendelivered

55

ENQA Workshop Reports

b. Funding: government has obliged H.E.-institu-tions to let the programmes be accredited beforethey will get funded

c. Mobility-enhancement: if Europe really wantsto be the most dynamic and innovative knowl-edge-economy then Europe should be devel-oped as one labour-market with a European la-bour-population. One of the essential prerequi-sites therefore is the enhancement of mobility.Accreditation is one of the possible ways to im-prove comparability between programmes.

d. Information facility: accreditation may be usedas a possibility for giving information to stu-dents, employers and the public. Of course, stu-dents and employers should be able to make adistinction between all different programmes,so for reasons of information only accreditationwould not be enough.

2. Much to my surprise, most stakeholders haveaccepted accreditation as a fact of life in a veryshort time. Of course, some objections were raised,especially from the universities and one particularpolitical party: they feared that institutions andprogrammes would only strive for the minimum-demands necessary for getting the accreditationdecisions. Eventually, one chose for the followingsystem:

a. Obligatory accreditation: necessary because ofthe funding-demand, each six years

b. Programme accreditation: two reasons for that:– A well-known institute should not necessari-

ly execute only good programmes– Deinstitutionalization of the binary system.

Universities were allowed to teach profes-sional programmes whereas universities ofprofessional education were allowed to teachacademic programmes (as long as they reachthe prerequisites for accreditation of thesespecific programmes).

c. Accreditation should be developed in line withthe well-known and broadly accepted DutchQuality Assurance system and it should not bedeveloped as a new bureaucratic system parallelwith the Q.A.-system. NAO decides on the basisof reports by quality agencies. All programmes

that want to deliver degrees acknowledged bythe Dutch government and want study-grantsfor their students younger than 30 years of ageshould be accredited. This means that all pro-grammes by public as well as private institu-tions have to be accredited.

d. All accreditation-decisions have to be madepublic

e. An appeal is possible

3. Netherlands Accreditation Organisation (NAO)has been established as an independent body inJune 2002.

a. Board of three fulltime members and four part-time – fulltime members are three former uni-versity presidents, who are resigned from theiruniversities. Part-time members are the formerminister of Education, two people with ampleexperience from industry, and one from the pub-lic sector.

b. Budget is 3.6 million euro pro year, whichmeans that programmes only pay a small fee foraccreditation – each accreditation-applicationcosts 2.500€.

c. Staff of 20 people, ranging from lawyers to ex-perts in all different disciplines

d. The NAO has developed frameworks for:– The accreditation of existing programmes– The advising on the perceived quality of new

programmes – eventually the minister takesthe decision whether or not a new programmemay start. These programmes must have hada positive advice by NAO.

– “Registration” of the Quality Agencies ofwhich NAO thinks that they can deliver goodand fair reports

– Some specific new programmes, for instancethe research-masters and programmes thatwant to enlarge the period of years a masterwill take.

4. Starting points

a. Respect for the field of H.E. No one is purposelypresenting a bad course. As former participantsin the field of H.E., we are convinced that most

56

ENQA Workshop Reports

programmes have a sufficient level to get ac-credited. We are not the accreditation police!

b. As close as possible cooperation or alliance withthe existing Q.A.-system to keep the improve-ment-function of the system going.

c. No more bureaucratic burden or high costs. Theburden of the quality assurance-system is al-ready high enough.

d. Development of a framework that is compatibleto international developments. We want our sys-tem and the framework to be in line with the in-ternational developments.

e. Close cooperation and dialogue with the insti-tutes of H.E. I strongly believe that the institu-tions themselves and their staff should have theconviction that accreditation is sound and fair.They are the ones that give the system the nec-essary legitimation.

5. The accreditation decision is dichotomous, it iseither yes or no; in the Netherlands there is noconditional or provisional accreditation. As aresult, there are four possible accreditation results:professional bachelor, academic bachelor,professional master and academic master.

Between academic and professional programmesexists a rather clear distinction:

• Differences in aim and goals of the course;• Differences in content;• Differences in quality of staff, esp. research ex-

perience;• Differentiation in relationship with the profes-

sional field: very strong in the professional ori-entation, weaker in the academic orientation.

The Accreditation Process

6. Accreditation is based on an application by aninstitution. The basis for the accreditation-decisionis a report by an external panel. The report has to bebased on the accreditation-framework, developedby our accreditation organization.

The external panel has to be appointed by – prefer-ably – a registered Quality Assurance Agency. Uptill now, five agencies have applied for registrationand three applications have been rewarded.

We have judged the agencies on 5 points:

a. The organizational and financial independence;b. Their competence to compose panels of the re-

quired quality and diversity;c. Their guarantees of the independence of the

members of the panel and their way of conduct-ing the evaluation-process;

d. The operationalisation of the NAO-framework;e. Their competence to compose a domain-specif-

ic framework for validating the specific course.

Each institute or program is free to select anagency. They may do this on the basis of the price,proven quality, or method of executing the proc-ess. By and large, you might say this is a strategicdecision following the choice for a profile a pro-gramme has made.

7. Let me elaborate on the composition of thepanel. We demand:

• Disciplinary expertise;• Educational expertise;• Audit expertise;• A student;• International expertise or knowledge of the in-

ternational developments in the field (where ap-propriate);

• Professional expertise/expertise from the pro-fessional field (where appropriate).

Of course one person may combine several exper-tises, but we obliged a panel to be composed atleast of 4 persons (of which one is a student). Thisis called the GOD-criteria: Gezaghebbend, Onaf-hankelijkheid en Deskundigheid – Authority, Inde-pendence and Expertise.

8. The panel will execute their work on the basis of:

• Desk research: a thorough examination of a pro-gramme’s self study, self-evaluation or manage-ment review;

• Site visit of two days, in which they will see theprogramme management, teaching staff, stu-dents, facilities, the examinations, final thesesand if appropriate alumni and employers.

57

ENQA Workshop Reports

The judgement must be presented to staff and man-agement, who – hopefully – will use the report toimprove the course-quality and to apply foraccreditation by NAO.

9. The NAO-framework

The framework has been composed of 6 subjects,21 aspects and 30 criteria. Deliberately, they aremade open, so that programmes themselves, thepanels or the quality assurance agencies are able tooperationalise them on the basis of their needs andwishes. Of course, the report that is part of theapplication must explain how the criteria are oper-ationalised. Generally, the Q.A. use a generalframework (which they presented to us duringtheir application for registration), but they arecompleted with domain-specific criteria.

The subjects have to be judged as sufficient ornot sufficient. In order to receive a positive accred-itation all six subjects must be judged as “suffi-cient”. Each subject consists of a number of as-pects, varying from two to eight. These aspectshave to be judged on a four point scale: insuffi-cient, sufficient, good or excellent. This has beendone to find “best practices” and to give the panelthe possibility to weight the different aspects. An“insufficient” on one aspect may be countered bya “good” or an “excellent” on another aspect, sothat the subject itself might be valued “sufficient”after all. Panels have to give an argumentation fortheir judgements on aspects and subjects.

10. Special features

a. Extraordinary elements of quality. These maybe part of the accreditation report, although theydon’t have any influence on the accreditationoutcome: it is an extra.

They may be for instance:

• Pedagogical system (for instance ProblemBased Learning);

• Internationalization (composition of staff andstudents);

• Excellent relation with the workfield;• Excellent quality.

Also, these special features have to be judged bya panel, in order for the NAO to validate the claimfrom the institute or programme.

11. International relations

a. From September 3, 2004 onwards, NAO is to bethe accreditation organization for the Flemishpart of Belgium as well. An agreement betweenthe two governments has been reached. Nether-lands Accreditation Organization will be theDutch-Flemish Accreditation Organization.Two more full time members of the board, fourmore part-time members; staff will be enlargedwith nine more people. The H.E.-systems inNetherlands and Flanders look largely the same,but there are differences, which have to be takeninto account in the frameworks and the proce-dures.

b. ECA: officially founded in Cordoba, November2003. – Goals: to understand and improve each oth-

er’s stand on accreditation to get “mutual rec-ognition” of accreditation decisions.

– Membership: officially recognized agenciesthat work with accreditations or accredita-tion-like procedures: Germany: Akkreditierungsrat and organiza-tions that have been recognized by theAkkreditierungsratAustriaSwitzerlandIreland: HETACNorway: NOKUTSpain: ANECANetherlands/Flanders: NVAO

– Five working groups: “mutual recognition”,“common framework of qualifications”,“publication of accreditation results”, “Min-isters’ conference in Bergen”, and “develop-ment in the field of accreditation”.

58

ENQA Workshop Reports

2.9 NORWAY: Quality Assurance of Norwegian Higher Education

Tove Blytt Holmen, Deputy Director General, Norwegian Acency for Quality Assurance in Edu-cation (Nokut), [email protected]

The higher education sector in Norway

The majority of Norwegian higher education insti-tutions are owned by the state: four universities,six specialised institutions at university level, twoacademies of fine art and 26 regional universitycolleges. More than 90 per cent of the student pop-ulation in Norway attend state institutions.

A few specialised institutions at university levelare private. The private sector is otherwise madeup of a number of fairly small institutions. Themain academic fields of private higher educationare business and management, ICT studies, theol-ogy, nursing and health care and teacher educa-tion.

Legal and regulatory framework; the ‘quality reform’ (2002)

Norwegian higher education is regulated by twolaws:

• The Universities and Colleges Act regulatesstate-owned institutions and their right to estab-lish programmes and award national degrees.This law also regulates the quality assurance ofhigher education.

• The Private Colleges Act regulates private insti-tutions’ right to award national degrees and theiraccess to public funding. Benchmark for therecognition of private higher education has hith-erto been corresponding provision in institu-tions under the Universities and Colleges Act.

Both laws were recently amended (2002) in con-nection with the Government’s ‘quality reform’ ofhigher education’. The amendments represent thefirst stage in a process with the aim of merging thetwo laws into one and thus create greater equalitybetween state and private institutions.

The reform process also:

• changed the degree structure in accordance withthe recommendations of the Bologna Declara-tion

• increased institutional autonomy• imposed a stricter obligation for institutions to

follow up students actively • introduced a system of formal accreditation for

all higher education• imposed stricter demands in the field of quality

assurance.

Quality assurance at the institutional level: internal quality assurance systems

Each institution is responsible for the quality of itsown educational provision. There is nothing newin this responsibility as such, but the institutionswill now be required to demonstrate how responsi-bility for quality is followed up with actual qualityassurance. After the reform, a prerequisite for thestatus of an accredited institution will be the exist-ence of an internal system of quality assurance thatcomplies with nationally set criteria. The institu-tions are expected to have such systems in place by1 January 2004.

Quality assurance at the national level: NOKUT

The Norwegian Agency for Quality Assurance inEducation (NOKUT) takes care of quality assur-ance at the national level. The agency, replacingthe former Network Norway Council, becameoperative on 1 January 2003. Unlike its predeces-sor, NOKUT is not a part of the government struc-ture and acts independently inside a given frame-work of law and a Ministerial Regulation. Its maintasks are to:

• make all accreditation decisions concerninghigher education that go beyond the institutions’self-accrediting powers. These decisions cannotbe modified by any other authority.

• evaluate and pass judgement on the institutions’internal quality assurance through quality au-dits, carried out in regular cycles and includingall accredited institutions. In addition to act as a

59

ENQA Workshop Reports

control mechanism, the audits are conducted ina way that is conducive to quality enhancement.

• carry out evaluations with the purpose of revis-ing specific accreditation. Any institution canhave accreditations revoked or suspended – forthe entire institutions as such, or for individualprogrammes – following a negative assessmentin this type of evaluation.

• carry out other types of evaluations with thegeneral purpose of investigating, assessing anddeveloping the quality of higher education inNorway. The Ministry may instruct NOKUT toundertake such evaluations.

• issue general recognition – or credit count to-wards national degrees – to higher educationfrom other countries, or to any other educationthat is not regulated by the Universities and Col-leges Act or the Private Colleges Act. This is apower it shares with accredited institutions.

Accreditation

As from 1 January 2002 accreditation is manda-tory and universal for all formally recognisedhigher education in Norway. Accreditation is notlimited to a specified period of time but will beconsidered as valid until explicitly revoked, fol-lowing an assessment. The new accreditation for-mula combines institutional and programme/course accreditation:

Institutional accreditation gives universities andcolleges certain rights to award national degrees ordiplomas.• When the law amendments became operative on

1 January 2002, all state-owned institutionswere automatically given status as accreditedinstitutions with certain degree-awarding rights.These rights vary with institutional category, ofwhich there are three: – University: (Full awarding rights at all levels,

including doctoral programmes);– Special-field university: (Full awarding

rights at all levels within given special field);– College: (Full awarding rights at the bachelor

degree level).• No private institutions were automatically giv-

en status as accredited, but may obtain it

through a process of institutional accreditation,for which a national set of standards has beengiven. Applications can be made for any of thethree categories and by May 2003 two privateinstitutions have applied, one to become a fulluniversity and the other to become a special-field university.

• By the same standards, state institutions mayseek accreditation in a different (‘higher’) cate-gory, following a process of institutional ac-creditation. By May 2003 one such applicationhas been registered, where a special-field uni-versity seeks status as full university. Three uni-versity colleges have declared their intention toapply for university status.

Programme accreditation may be obtained forspecific courses or programmes that the institutionis not institutionally accredited to provide.

• All higher education programmes and courses –in state or private institutions – that were recog-nised under the previous guidelines by 1 Jan.2003 were automatically given status as accred-ited.

• New provision in accredited institutions thatgoes beyond the awarding rights that followfrom institutional category must be accreditedafter a process and in accordance with nationalsets of standards (e.g. master degree pro-grammes in institutions of the college category).

• All new (or not previously recognised) provi-sion in unaccredited institutions must undergosuch a process in order to become accredited.

Accreditation control

Accreditation control is carried out through a com-bination of institutional audits and specific pro-gramme or subject assessments, referred to as‘revision’:

• Institutional audits represent the systematic,comprehensive mechanism for external scrutinyof the quality of higher education. Institutionalaudits will be conducted in all accredited insti-tutions, irrespective of category, and there mustnot be more that six years between each time an

60

ENQA Workshop Reports

institution is evaluated. The frame of referencefor these evaluations is made up of the nationalcriteria that have been set for internal quality as-surance systems, which among other things de-mand that these systems must be able to detectinferior quality, and that quality assurance mustbe adequately documented. Failure to provideinternal quality assurance in accordance withthe criteria will have as a consequence that theinstitution cannot any more offer new provision.But the audits cannot in themselves lead to theloss of accreditation.

• Revision of accreditation will take the form of aspecific (programme, course, discipline, or eveninstitutional) assessment, directed at an identi-fied unit of education. Revisions will be trig-gered by indications from the audits, but also byother indications, and they may be carried out asrandom tests. A negative result will lead to theloss of accreditation, and consequently to theloss of public funding.

Standards and criteria for accreditation

a) Institutional accreditation:

General standards for institutional accreditation inany of the three categories are set by MinisterialRegulation. The Regulation authorises NOKUT todevelop a further concretisation of these standardsin the form of more detailed assessment criteria.The criteria, which have been developed after con-sultations with the sector, concern internal qualityassurance, academic competence levels, researchactivities, the number and levels of existing pro-grammes, internationalisation, infrastructure andinstitutional organisation and management.

b) Programme accreditation:

For the accreditation of individual programmes,both the standards and the more detailed assess-ment criteria are set by NOKUT, again after con-sultations with the sector. These standards aregeneric descriptors for types of degree pro-grammes, as defined by their level and credit vol-ume (2-year programmes; bachelor programmes;master programmes; doctoral programmes.) Spe-cific national quality criteria for the different disci-

plines or subject areas have not been defined. Itshould be noted, though, that a few programmesaimed at professions (notably teacher education)are regulated in more detail by national curriculumguidelines.

Standards and criteria for internal quality assurance

A brief and general standard for the institutions’internal quality assurance systems is set by Minis-terial Regulation, indicating what these systemsshould include and achieve. This standard, in turn,has been further developed by NOKUT through aset of assessment criteria that will be applied in theinstitutional audits. Institutional audits are not for-mally accreditation procedures and the assessmentcriteria are not the same as absolute standards.

The proposed criteria are developed from twobasic considerations: that quality assurance sys-tems should aim at both control and enhancementand that the criteria should focus on system qualityrather than on any specific quality assurance meth-ods or measures.

The criteria focus on (i.a.) the following fea-tures:

• That the QA system is linked to institutionalsteering and management and made an integralpart of the institution’s strategic work.

• That quality work is organised in routines andmeasures that ensure broad participationthroughout the institution, with defined distribu-tion of responsibility and authority.

• That documentation from all study units (in-cluding franchised provision) is sufficient, andthat it always includes the students’ assessmentof the programmes they attend.

• That documentation is analysed, summed upand reported.

• That quality assurance and quality work is ac-counted for in annual reports to the institution’sboard.

All standards and criteria for accredita-tion and the evaluation of quality assur-ance systems are presented in extensoon NOKUT’s website: www.nokut.no.

61

ENQA Workshop Reports

3 ConclusionKo Scheele, Inspector, Inspectorate of Education in the Netherlands, [email protected] Hämäläinen, Coordinator, ENQA, [email protected]

Introduction

In the Berlin Communique of 2003 the EuropeanMinisters pay high attention to higher educationquality assurance. They expect by 2005 the qualityassurance agencies to fulfil action “to develop anagreed set of standards, procedures and guidelineson quality assurance, to explore ways of ensuringan adequate peer review system for quality assur-ance and/or accreditation agencies or bodies”.Moreover they challenge the national higher edu-cation systems: “By 2005 national quality assur-ance systems should include:

• A definition of responsibilities of the bodies andinstitutions involved;

• Evaluation of programmes or institutions, in-cluding internal assessment, external review,

participation of students and publications of theresults;

• A system of accreditation, certification or com-parable procedure;

• International participation, co-operation andnetworking.

The ENQA workshop focused on accreditation inEurope. About 50 experts of 17 countries con-cluded that the two days of work in Rome wereable to cover in detail both the above-mentionedEuropean and various national perspectives ofaccreditation, described elsewhere in the report.

Accreditation concepts, scope and working methodsDue to the European ministers’ inclusion ofaccreditation into their Berlin Communiqué of

accreditation audit assessment external ex

provider programme learner output

governance& regulation

content ofprogrammes

curriculumdesign

financialviability

learningexperience

qualification

medium ofdelivery

adminsupport

studentsupport

organisationalprocesses

documentanalysis

stakeholdersurveys

directintervention

proxydelegate

accountability control compliance improvement

self-assessment PIs peer visit inspection

Approach

Object

Focus

Rationale

Methods

External evaluation cre

62

ENQA Workshop Reports

2003, this widely used method of quality assurancehas become an important part of European qualityassurance framework, and accrediting countrieshave accordingly increased their mutual coopera-tion in order to learn from one another and to shareexperiences. The European accreditation Associa-tion ECA, consisting of accreditation agencies ofvarious countries was recently established(November 2003, Cordoba).

Accreditation in this report fits as a tool in thebroader concept of higher education quality assur-ance. The contributions showed a large variety onaccreditation concepts, scope and working meth-ods. Accreditation is a used both to validate insti-tutions and study programmes. Sometimes it isused only in the case of new study programmes,while in other countries all study programmes are(to be) accredited.

Despite the differences, it is interesting to noticethat there is convergence, not due to homogenisa-tion but due to comparable use of the various in-struments. Moreover: a national agency uses vari-ous instruments varying along the evaluation ob-ject, the focus and the rationale. Also, there is achange of approach visible after several years ofevaluation e.g. accreditation after a decade of pro-gramme-assessment, or programme accreditation

after a decade of audits. This, of course, is due tothe phase of evaluation and evaluation culture inthe given country. This tailor-made approachseems valuable as it responds to the rapidly in-creasing variety of higher education.

One of the conclusions was that recently someconsiderable effort has been put into the develop-mental aspect of accreditation, that is, accredita-tion will be more and more used to issue recom-mendations to the evaluated institutions and pro-grammes, as compared to an earlier concentrationon definitive yes/no decisions. Conditional accred-itations are also used, thus bringing the concept ofaccreditation closer to regular evaluations, whereencouragement for institutional development is acrucial objective. Qualitative accreditation has be-come increasingly visible beside the quantitativeone. An important subject is the focus of accredi-tation. The workshop discussed the aim of learningoutcomes, in particular the distinction between fit-ness of purpose (the goals and aims) and fitness forpurpose (the way these were made practice to-wards and the effects on students). The Europeanqualification framework given in the BolognaDeclaration is a good starting point. Agencies andthe higher education institutions are challenged toprioritise this. The so-called Dublin descriptors are

accreditation audit assessment external ex

provider programme learner output

governance& regulation

content ofprogrammes

curriculumdesign

financialviability

learningexperience

qualification

medium ofdelivery

adminsupport

studentsupport

organisationalprocesses

documentanalysis

stakeholdersurveys

directintervention

proxydelegate

accountability control compliance improvement

self-assessment PIs peer visit inspection

Approach

Object

Focus

Rationale

Methods

External evaluation cre

63

ENQA Workshop Reports

also of use, but they need refinement in the givennational context and discipline.

The discussion on accreditation also highlightedcritical success factors:

• The accreditation costs are seen as investments;• Accreditation is of use for the internal quality

assurance mechanisms;• Accreditation effects are in line with the costs

(both in time and money) e.g.:– Quality enhancement noted by students and

employers/society;– Mobility of students as result of transparent

accreditation decisions;– Few appeals.

The workshop discussed the dilemma of how tobring more objectivity into accreditation systemsso that decisions are, and are also acknowledgedby the accredited institutions, made according to ascomprehensive an amount of information as possi-ble. Attention was given to peer review as basis ofaccreditation:

• Renew peer review panels periodically;• Allow young professors to enter into the panels;

it is not always obvious that high ‘ranked’ uni-

versity professor are interested in undergraduateeducation;

• Training of peers by the agency is important;• The agency should facilitate the panel during

the whole procedure, including update informa-tion

• International peers are of use, but be aware ofthe threat of ‘cultural imperialism’ (especiallythe Anglo Saxon countries have a language ad-vantage);

• Use existing data of the given institution andvary the method; mind large self evaluation pa-pers (which are descriptive and not analyticaland not self-critical).

A notice was also given to the fact that in manycountries accreditations are in fact based on evalu-ation decisions, showing clearly that it may be rel-evant to address in more detail synergies betweenevaluation and accreditation in a follow-up work-shop, for instance.

And last but not least, take your own medicine:‘practice what you preach’! Not only transparentprocedures and reports are useful, an external eval-uation of the agencies themselves would be of val-ue for the recognition of the system.

64

ENQA Workshop Reports

Appendix

WORKSHOPACCREDITATION MODELS IN HIGHER EDUCATION

EXPERIENCES AND PERSPECTIVESItaly, Rome, 13–15 November 2003

GOALS� Understanding the principles of the various accreditation methodologies and models in higher education� Examples of possible good practices on the basis of country case studies� Discussing and evaluating usefulness of accreditation activities, strengths and weaknesses� Recommendations for the future

WORKING METHODS: KEYNOTERS, PLENARY DISCUSSIONS, COMPARATIVE WORKSHOPS AND A PROBLEM MARKET

1. Context of accreditation

� Background for accreditation in European Higher Education� Range of definitions used for accreditation� Range of methodological approaches to accreditation in higher education

An inventory “problem market” will be implemented dealing with critical issues:� Mutual accreditation policies� Strengths and weaknesses of the various approaches� …

2. Country case studies

Presentations on the following issues:� Scope of accreditation� Models, methods, targets and procedures of accreditation� Achieved results, experiences, and lessons learned

3. Results and perspectives

� Results, conclusions and recommendations of the workshops� Future perspectives

TARGET GROUPS

� Staff of ENQA member agencies� Accreditation & Quality Assurance experts suggested by ENQA member agencies� Ministerial Representatives of ENQA

65

ENQA Workshop Reports

PROGRAMME

Thursday, 13 November 2003

15.00 – 16.00 Registration and welcome coffee

Opening

16.00 – 16.30 Opening remarks, Luigi Biggeri, CNVSU Vice President

Welcome words, Alessandro Bianchi, CRUI

Kimmo Hämäläinen, ENQA Coordinator

Plenary Session

Chair: Luigi Biggeri

16.30 – 17.00 An approach to accreditation: the path of the Italian Higher Education

Carlo Calandra Buonaura, CNVSU Board

Primiano Di Nauta, CNVSU

17.00 – 17.15 Discussion

17.15 – 17.45 Accreditation, an irregular perspective

Ko Scheele, The Inspectorate of Higher Education, The Netherlands

17.45 – 18.00 Discussion

18.00 – 18.30 Inventory problem market

Ko Scheele

18.30 Cocktail

Friday, 14 November 2003

09.30 – 10.00 ENQA in the Bologna Process

Christian Thune, ENQA Chairman

10.00 – 10.30 Discussion

10.30 – 11.00 Coffee break

Parallel workshops

13.00 – 14.45 Lunch

Plenary Session

14.45 – 15.15 The Power of Accreditation: views of academics

Lee Harvey, Centre for Research and Evaluation, Sheffield Hallam University, Director, UK

15.15 – 15.45 Discussion

Moderator Workshop I Helmut Konrad

Workshop II Angelika Schade

Workshop III Ko Scheele

11.00 – 11.40 Introductions:scope of accreditation

� accreditation in the national context

� new programmes vs. whole system

� public and/or private institutions

� institutional or programme approach

� consequences

NorwayTove Blytt Holmen

AustriaKurt Sohm

GermanyHans-Uwe Erichsen

ItalyAlessandro Figà-Talamanca

NetherlandsKarl Dittrich

HungaryKrisztina Rozsnyai

11.40–13.00 Group discussion Group discussion Group discussion Group discussion

66

ENQA Workshop Reports

15.45 – 16.00 Coffee break

Parallel workshops

18.45 Rome by night sightseeing and dinner

Saturday, 15 November 2003

Plenary Session

Chair: Luigi Biggeri

09.30 – 11.00 Accreditation concepts, scope and working methods: outcome from the three workshop sessions

Angelika Schade, Ko Scheele, Kurt Sohm

11.00 – 11.30 Discussion

11.30 – 11.50 Coffee break

11.50 – 12.50 Problem market

Ko Scheele, The Inspectorate of Higher Education, The Netherlands

12.50 – 13.00 Closing remarks

Luigi Biggeri, CNVSU Vice President

Moderator Workshop I Kurt Sohm

Workshop II Angelika Schade

Workshop III Ko Scheele

16.00 – 16.40 Introductions: working methods

� commissions/panels, peers, working field, students, stakeholders

� frames of reference� information bases� procedures � report

NorwayTove Blytt Holmen

GermanyHans-Uwe Erichsen

HungaryKrisztina Rozsnyai

AustriaHelmut Konrad

NetherlandsKarl Dittrich

ItalyMuzio Gola

16.40 – 18.00 Group discussion Group discussion Group discussion

67