View
221
Download
5
Category
Preview:
Citation preview
QUALITYASSURANCEINHIGHEREDUCATIONINEASTERNEUROPE,CAUCASUS,CENTRALASIA
ANDMONGOLIA
TheStateofAffairsandtheImpactofQualityAssuranceandAccreditationonTertiaryEducationinEasternEurope,Caucasus,CentralAsia,andMongolia
SCOPINGSTUDY
REPORT
March2014
ReportonaScopingStudyofQualityAssuranceinHigherEducation
2
FOREWORDThe International Network of Quality Assurance Agencies in Higher Education(INQAAHE) has undertaken the current scoping study under the auspices ofUNESCO.INQAAHE had already in the past been cooperating with UNESCO in theframework of the so‐called Global Initiative for Quality Assurance Capacity(GIQAC) project. In continuation of this collaboration, INQAAHEhas nowbeencommissioned to execute a Scoping Study to better understand the qualitydevelopmentsinthecountriesofEasternEuropetheCaucasusaswellasCentralAsia (including Mongolia). More specifically, the study aims at identifyingdevelopmentaltrendsandgoodpracticesinthesethreeregionswhileprovidingrecommendationsforfutureimprovementandenhancement.TheScopingStudywas launchedonNovember29th,2013andendedonMarch31st,2014.TheINQAAHEBoardsetupateamofexpertstoconducttheScopingStudy.Theteamcompositionisasfollows:
‐ Teamleader:SusannaKarakhanyan,PhD,INQAAHEVice‐President‐ IringWasser,INQAAHEBoardMember,CEENQAPresident‐ GaneshHegde,INQAAHEBoardMember‐ MilaZarkh,CEENQASecretariat
TheFinalReportwasdiscussedattheINQAAHEBoardleveltobringindiversityofinsightsofalltheBoardmembers–representingallthemajorregionsoftheworld–aswellastovalidatetherecommendations.ThisreportconstitutestheFinalReportandisinlinewiththeTermsofReferencetoINQAAHE.Theresearchteamextendsits gratitudetoallthose,whohavecontributedtothe
productionofthisreport:theexternalqualityassuranceagenciesintheEasternEurope,Caucasus,Central
AsiaandMongolia.
SpecialthanksandappreciationgotoDavidWoodhouseandLeeHarvey
whohavekindlycontributedtothereportproductionwiththeirinsightfulfeedback.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
3
TableofContents
EXECUTIVESUMMARY....................................................................................................4
I. INTRODUCTION..........................................................................................................9
II. GENERALANDSPECIFICOBJECTIVESOFTHESTUDY.................................11
III.METHODOLOGY......................................................................................................12
IV.MAJORDEVELOPMENTSINQUALITYASSURANCEINEASTERNEUROPE,
CAUCASUS,ANDCENTRALASIA................................................................................13
1.GeneraloverviewofQAsystems...................................................................................13
2. QualificationsFrameworkandLearningOutcomes...........................................15
3. TheQualityAssuranceProvider,GovernanceFrameworkandMission......16
4. HumanandFinancialCapacity...................................................................................17
5. ExternalEvaluationProcedures................................................................................18
6. QualityAssessmentCriteria:developmentandimprovement.......................23
7. Accountabilitymechanisms........................................................................................24
8. StakeholderInvolvement............................................................................................25
9. CapacityBuilding............................................................................................................26
10. Networking....................................................................................................................27
V. CONCLUSIONS..........................................................................................................28
VI.APPENDICES.............................................................................................................30
AppendixA:ListofAcronyms.............................................................................................30
AppendixB:ListofParticipantCountriesandAgencies............................................31
AppendixC:ContactdetailsofParticipatingQAAgencies........................................33
ReportonaScopingStudyofQualityAssuranceinHigherEducation
4
EXECUTIVESUMMARY
he International Network for Quality Assurance Agencies in HigherEducation (INQAAHE) in line with its mandate of being an umbrellaorganization of all the quality assurance (QA) agencies and networks at
international level has been actively promoting the establishment andenhancement of quality culture in its member countries including EasternEurope, Caucasus, Central Asia and Mongolia. To better understand thedevelopmentaltrendsandtoidentifygapsaswellasgoodpractices,theScopingStudyontheStateofAffairsandtheImpactofQualityAssuranceandAccreditationon Tertiary Education in Eastern Europe, Caucasus, Central Asia andMongoliaundertheauspicesofUNESCOwasinitiated.Theobjectivesofthecurrentstudywereasfollows:
‐ To assess the situation of quality assurance organizations in Eastern Europe(South Eastern Europe), Caucasus, and Central Asia. It targets both countrieswith established QA entities aswell as countrieswith no such entities. In theabsenceofaformalentityforQA,thestudytargetedQAactivitiesinthecountry;
‐ To identify gaps in areas in the deployment of QA systems in the targetcountries;
‐ ToassesstheimpactofQAinthesub‐regionswitha focusonthe landscapeofQAintheregion(s).
‐ Toproviderecommendationsforfutureimprovementandenhancement.
As a result of the scoping study the followingmajor findings, subdivided inareas of good practice, areas for future improvement and relatedrecommendationshavebeenidentified:
EXTERNALREFERENCEPOINTS
GoodPractice
‐ Establishment of National Qualification Frameworks in most countries underreview;
AreasofConcern
‐ InmanyinstancesNationalQualificationsFrameworks(NQFs)areintheprocessofestablishmentand,insomesystems,toolsmeasuringandassessingalignmentofstudy/academicprogrammeswiththeseNQFshavebeenlaunched.However,mostofthesystemshavenotstartedtheprocessofalignmentyet;
‐ The external quality assurance frameworks of some of the countries areabundantwithmultipleQAtools—institutionalregistrationandaccreditation,programme evaluation and accreditation, attestation, monitoring supervision
T
ReportonaScopingStudyofQualityAssuranceinHigherEducation
5
etc.Frequentlytherearenoprovisionsinplacetousesynergiesbetweenthesevarious forms of QA, duplicating work and increasing the burden on highereducationinstitutions(HEIs).
Recommendations‐ To ensure a coherent, complementary and consistent approach to external
qualityassurancethecountriesinthesampleshouldreviseandcoordinatethevarious external quality assurance mechanisms in place at the national level.Such a revision should aim at enhancement of HEI´s educational performancewhilealleviatingtheiradministrativeburden;
‐ Toestablishandoperationalizetoolsensuringalignmentofthestudy/academicprogrammeswiththeNQFsandcheckingsuchanalignment.
GOVERNANCEANDINDEPENDENCE
GoodPractice
‐ InclusionofinternationalexpertsintheGoverningBoardsoftheexternalqualityassuranceagencies(EQAAs),whichcontributes todiversityand impartiality indecision‐takingtoaconsiderableextentwhileatthesametimereducingconflictofinterests;
AreasofConcern
‐ Governments andministries’ interference in the external quality assurance ofHEIsandprogrammesinmanycountriesisstilltangible,oftentothedetrimentofHEI’sautonomy,accountabilityand,therefore,qualityofprovisions;
‐ Governanceoftheagenciesisstillinfluencedbytheparticipationofministryandgovernmentrepresentatives,who,insomeofthecasespresideinthedecision‐makingbodiesonanexofficiostatus.Ontheotherhand,someofthegoverningbodies include internationalexperts,which, tosomeextent,brings indiversityandimpartialityindecision‐taking;
‐ Independenceofdecision‐takingisnotensuredinmostofthecases;
Recommendations
‐ Reconsider the role of the ministries and governments by limiting theirinvolvement in independentexternal reviewtoallow for independentexternalqualityreviewanddecision‐making.
STAKEHOLDERINCLUSION
GoodPractice
‐ Stakeholder inclusioninvariousqualityassuranceactivitieshasbeenobservedasamajortrend.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
6
AreasofConcern‐ Stakeholder involvement in quality assurance is in place to some extent. Such
major impediments as restrictions in the legal framework, lack of funding,scarcityofcapacitybuildingeventsanddisseminationactivitiesandtoalesserextent lack of inclusion in the decision‐making processes have however beenidentified; increased involvement of the EQAAs in the international networksseemstobeatrend.However,apro‐activeparticipationisstilltobedesired.
Recommendations‐ Devise and implement consistently policies and approaches in regard to
stakeholder involvement inexternalqualityassurancebyconveyingthevaluesandbenefitsoftheexercisetothestakeholders.
USEOFINTERNATIONALEXPERTS
GoodPractice
‐ Theexternalreviewers’panelsincludethekeystakeholders—facultymembers,students,andemployers—inmostofthecases;
‐ In some cases, there is representation of international peers on the externalreviewers’panelsandconductoftheproceduresintwolanguagesforincreasedrecognitionattheinternationallevel
EXTERNALQUALITYASSURANCESTANDARDSANDPROCEDURES
GoodPractice
‐ The external evaluation procedures follow the generally accepted stepsrevolving around self‐evaluation, desk‐review, site‐visit, panel reportproduction,decision‐takingbyrespectiveaccreditationcouncilsorcommissionsandfollow‐upconductedbytheHEIsandoverseenbytheEQAAs;
‐ Development of criteria in most of the cases is done by benchmarking theinternational good practice, identifying local needs and refining the criteria tolocal needs. Such criteria are ideally piloted prior to full implementation toensurelegitimacyoftheadoptedapproachesandrelevancetothecontext;
AreasofConcern‐ In someof thecasesadirect transferof criteriawithout further refinement to
thelocalneedshasoccurredleadingtothedamagetothesystem;
Recommendations‐ Whilerevisingthecriteria/standardsforexternalevaluationthefollowingsteps
shouldbeundertakentoensurethecriteriadomeettheneedsofthesystemandlead to its enhancement: (1) benchmarking with international trends; (2)identificationof localneeds;(3)developmentof thedraftcriteria/standards inbroad consultation with the stakeholders; (4) pilot of the criteria/standardspriortofullimplementation;(5)refinementandofficialadoption.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
7
ACCOUNTABILITY
GoodPractice
‐ Within the EQAAs development and operationalization of such accountabilitymechanisms,asinternalqualityassurancemechanismhasbeenobservedandiscommendable.
‐ Some well‐established agencies have undergone external recognition againstinternationally accepted standards (ENQA ESGs), which contributes to theirinternationalstandingandcredibility.
‐ IncreasedinvolvementoftheEQAAsintheinternationalnetworksseemstobeatrend.
AreasofConcern‐ Whilesomeoftheagencieshaveacertainnumberofaccountabilitymechanisms
inplace,mostofthemarestill intheprocessof introducingandrefiningthem.Development andoperationalization of internal quality assurancemechanismsat the EQAA level seems to be not adequately institutionalized inmost cases.Furthermore, such exercise as external recognition of the agency againstinternationally accepted standards (ESGs,GGP, ChibaPrinciples) is atminimallevelinsomesystemsunderreview.
Recommendations
‐ Develop and implement a robust internal quality assurance system withinEQAAs to ensure continuous improvement of the services as well asaccountability.
‐ MakeusebyEQAAsofinternationallyacceptedstandards(e.g.ESGs,GGP,ChibaPrinciples) to enable international recognition of EQAAs against thosestandards.
FINANCIALANDHUMANRESOURCECAPACITY
AreasofConcern
‐ Capacity building of the EQAA staffmembers is essential to the successof theexternal evaluation endeavour. Also the role of the staff varies considerably,someusingthemmainlyforadministrativepurposes,somegivingthemamoreprominent/responsibleroleintheQAendeavours.Mostofthesystemshavealsoinadequatestaffingtoensuresystem‐wideresearch,identificationoftrendsandfurtherenhancementofthesystem;
‐ Amajorneedincapacitybuildingofexternalevaluatorshasbeenrevealed.Suchneedsmainlyevolvearoundinterpretationofthecriteria/standards,evaluationbased on the criteria/standards, production of the final report by the panel,development of recommendations that are clear enough and valid as well asclearapproachestofollow‐uptopromotesystemenhancement;
ReportonaScopingStudyofQualityAssuranceinHigherEducation
8
‐ Financial capacity of the EQAAs in the target countries is at risk because ofunderfundingfromthegovernmentandlackofalternativerevenuesources.
Recommendations‐ Engage in major capacity building activities for external reviewers to ensure
adequateinterpretationofthecriteria/standards,necessaryskillsinevaluationbased on the criteria/standards, production of the final report by the panel,developmentofrecommendationsthatareclearenoughandvalid.
‐ EnsurecontinuousprofessionaldevelopmentandcapacitybuildingfortheEQAAstaff to enable them to participate more actively in the external qualityassurance reviews and to perform such major functions as system‐wideresearch, identification of gaps and developmental trends and creation ofrecommendationsforimprovementandenhancement.
‐ Revise the approaches to funding mechanisms to enable cash flow fromalternativesourcessuchasconsultancyandcapacitybuildingservicesaswellasimplementation of international and local grants to be less dependent on thegovernmentalallocations. Itgoeswithoutsayingthatdiversificationof incomesourcesmustnot infringeon thequality and independenceof the externalQAdecision‐makingprocess.
‐ Ensurepro‐activeparticipationofEQAAsandHEIsintheinternationalnetworksandtakeadvantageofinvolvementindevelopmentalactivitiescarriedoutbythenetworks.
‐ ***‐
ReportonaScopingStudyofQualityAssuranceinHigherEducation
9
I. INTRODUCTION
ntandemwithglobalizationandmassificationthehighereducationsystemsin EasternEurope, Caucasus and Central Asia, like all other systems in theworld, experienced mushrooming of a diversity of HE providers, some of
them with a questionable quality. As a result, perspectives on the quality ofprovisionsandrelevanceoftheawardedqualificationstothemarketneedswereat risk. Thus, the countries in Eastern Europe, Caucasus and Central Asiaembarkedona journeyof rethinking thearchitectureof their systems toalignwith international trends, therebyenhancing thequalityandrelevanceof theirprovisionsaswellasbeingvisibleintheinternationalarena.While some countries (e.g. USA) have enjoyed a history of over a century inexercising quality assurance in HEIs in a systemicway, the cases in the threesub‐regionsarenewlyemergingones.UnlikeindecentralizedsystemswheretheinitiativeforexternalqualityassuranceingeneralandaccreditationinparticularcamefromtheHEI leaders(e.g. theUSAandNewZealand)the initiativeof theestablishmentofQAsystemsinthetargetregionspredominantlycamefromtherespectivegovernments intheirsearchforbettermechanismstosafeguardthequalityofHE.Being concernedabout thenature andperformanceofHEIs andtheir impact on the economic development, respective governments, in theirsearchtoestablishmeasuresforaccountability,tookdecisionstoestablishsuchexternalmechanisms as licensing and accreditation. In some systems it took aformofaquasi‐autonomousQAagency, inothers thegovernmentsestablishedunitswithin respectiveministries to take upon themission of external qualityassuranceinanefforttokeepcontrol.One of the changes that most affected the higher education sphere was theBolognaDeclaration in1999withagoal to increaseacademicandprofessionalmobilityonthebasisofrecognizedandtransferablequalifications.Thelaunchofthe Bologna process became also an impetus to a revised approach to qualityassurancebyemphasizinganindependentandenhancement‐drivennatureofitby many of the countries in the three regions. Educational policymakers inEastern Europe, Caucasus and Central Asia, along with their Europeancounterparts,identifiedtoolsthatcouldbeusedtopromotethecommongoalsofrecognitionandpromotionofprofessionalandstudentmobility.Thus,theneedto establish independent and robust external and internal quality assurancemechanisms to attest to the public about the validity of theHE Providers andcredentials was tangible. Consequently, most of the countries in the areaembarkedon establishment of quality assurancemechanisms, both at nationalandinstitutionallevelsandbroadlyexercisethem.The International Network of Quality Assurance Agencies in Higher Education(INQAAHE)inlinewithitsmandateofbeinganumbrellaorganizationofalltheQA agencies and networks at international level has been actively promotingestablishment and enhancement of quality culture in its member countries
I
ReportonaScopingStudyofQualityAssuranceinHigherEducation
10
including the 3 sub‐regions. Thus, to better understand the developmentaltrends,identifythegapsandgoodpractices,thisScopingStudywasinitiated.Asa result of this study a set of recommendations is addressed to furtherimprovementandenhancementofthetargetsystems.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
11
II. GENERALANDSPECIFICOBJECTIVESOFTHESTUDYTheobjectivesof theScopingStudyontheQualityAssurancestateofaffairs intheEasternEurope,CaucasusandCentralAsiaareasfollows:
1. AssessthesituationofqualityassuranceorganizationsinEasternEurope(South
Eastern Europe), Caucasus, and Central Asia. It targets both countries with
establishedQAentitiesaswellascountrieswithnosuchentities.Intheabsence
ofaformalentityforQA,thestudytargetedQAactivitiesinthecountry;
2. IdentifygapsinareasinthedeploymentofQAsystemsinthetargetcountries;
3. AssesstheimpactofQAinthesub‐regionswithafocusonthelandscapeofQAin
theregion(s).
4. Providerecommendationsforfutureimprovementandenhancement.
Theexpectedoutcomesevolvearound:
1. BaselineinformationthatwillgiveabetterunderstandingoftheQAsituationin
theEasternEurope,CaucasusandCentralAsiaincludingMongolia;
2. Identifiedissues,challengesandgapsinQAintheEasternEurope,Caucasusand
CentralAsia;
3. IdentifiedQAgoodpracticesthatshouldcontributetotheimprovementofaQA
cultureintheEasternEurope,Caucasus,CentralAsiaandMongolia.
4. Recommendationson furtherenhancementofqualityassurance in theEastern
Europe,Caucasus,CentralAsiaandMongolia.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
12
III. METHODOLOGYThemajormethodsforundertakingtheScopingstudyevolvedaround:
‐ AsurveyaddressedtothebodiesthatarecarryingoutfunctionsofqualityassuranceinthecountriesofEasternEurope,CaucasusandCentralAsia(Mongoliaisalsoincludedinthesample).Thequestionnairedesignedforthisstudyintendedtolookatsuchmajordevelopmentsinthesphereofquality assurance asnatureof theQAprovider, governance frameworkand mission, human and financial capacity, the external evaluationprocedures undertaken by the bodies, the QA criteria and theirdevelopment and improvement nature, accountability mechanisms,stakeholder involvement, capacity building events undertaken by therespective bodies to enhance the system as well as the networkingcapacityofthosebodies.
‐ Analysisofthedataobtainedfromwebsitesoftherespectivebodiesandinternationalnetworks.
The survey was distributed to 33 bodies charged with quality assurancefunctionsin28countries(seeAppendixBandCforthelistofthecountriesandparticipatingagencieswithcontactdetails).The questionnaire was distributed electronically via eSurvey athttps://www.esurveycreator.com. Both the paper based questionnaire and thelink to theonlinesurveyweresent to34agencies/ministries in29countries inEasternEurope,CaucasusandCentralAsiaincludingMongolia.Wehavereceived23(67.64%) responses from 19 (65.5%) countries. Almost all the qualityassurance agencies responded. The countries, where the QA function is stillperformedbytheministriesofeducation(27.5%)didnotrespondtothequery.Alanguagebarriermighthavebeenamajorfactorforthenon‐response.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
13
IV. MAJORDEVELOPMENTSINQUALITYASSURANCEINEASTERNEUROPE,CAUCASUS,ANDCENTRALASIA
1.GeneraloverviewofQAsystemsIfwehaveanoveralllookatthethreeregions,outof29countriesinthetarget8(27.5%)didnothaveQAagencies and theministriesof education and scienceperformed the quality assurance functions. Below is the distribution of QAfunctionsperregionandjurisdiction:
Chart1:DistributionofQAfunctionsperregionandjurisdictionAs seen from Chart 1, in Central Asia (in spite of the existence of the CentralQualityAssurancenetworksandongoingactivitiestocreatenewQAentities,e.g.intheframeworkofthecurrentQUEECAproject)andtosomeextentinEasternEuropeandCaucasus,ministriesarestilltryingtokeepcontroloverthesystem,thus implementingthequalityassurancefunctions. Inthesesystems,accordingtotherespondents,independentEQAcontinuestobecompromised.ContinuousreformofthesesystemsisneedediftheEQAistobeoperationalizedsuccessfullythus leading to system enhancement, relevance and, therefore, internationalvisibility in the long run.However, the legacy of a centrally controlled systemshould be kept inmind, while revising the approaches to governance, since atotallyindependentandprivately‐runmodelofEQAAseemstobelessfeasibleinsuch contexts. The primary reason for such a concern is being under thejurisdiction and protection of the respectiveministries and already entitled toconferdegreesthepublicHEIsareactuallyreluctanttoundergoproceduresthatarenotstipulatedbythegovernment.As for the age and extent of maturity of the EQAAs, establishment of suchagencies is at various levels in the target countries. Some havemore than 20years of experience being established in the early 1990s, others are newlyemerging ones established in 2011. However, with regards to maturity some
Caucasus EasternEurope CentralAsia+Mongolia
3
18
215 5
QAA Ministry
ReportonaScopingStudyofQualityAssuranceinHigherEducation
14
systems, albeit the history of over 15 years, are still being challenged andexplicitlystateaneedforamajorrevisionoftheapproaches.Most agencies in the scopewere theonly agencies at the country level todealwithqualityassurance.In7countriesoutof19thatresponded,however,morethanoneagencywasoperating–bothatthesubjectspecificandgenerallevels.91.3%of the respondentsareoperating ina single countryandonly13%areoperatinginternationallyaswell.Further,inthecountriesthatrespondedtothequestionnaire there are 61 quality assurance agencies, out of which 28 aresubjectspecificand33generallyoperatingagencies.Alltheagenciesinthesamplecoversuchtypesofprovidersaspublicandprivateuniversities. In addition to the public and private HEIs, 13 agencies (56.5%)covernon‐universityprivateandpublicprovidersand4agencies(17.4%)covertransnationalprovidersaswell.Further, almost all the respondents are involved in one ormore internationaland regional quality assurance networks. The distribution of networkmembershipsisasreflectedbelow:
Chart2:Membership/affiliationtonetworks1AsseenfromChart2,thepopularityofsuchagenciesasCEENQA,INQAAHE,andENQAisstriking.APQNisalsopopularinthethreeregionstosomeextent.Furthermore,ENQA(ESGs),INQAAHE(GGP),APQN(Chibaprinciples)alsohaveestablished standards against which they conduct external evaluation of theagencies for recognition purposes and ENQA also does it for granting amembershipstatus.Amongtheagenciesthatrespondedtothesurvey,7agenciesareENQAfullmembersoutofwhichthetermforoneoftheagenciesexpiredin2012. The rest are ENQA associates, which means they are still supposed toundergo external evaluation against ESGs to become full members. As for theINQAAHE GGP alignment or external recognition against the APQN Chiba
1SeeAppendixAforacronyms
0 5 10 15 20
INQAAHE
ENQAmember/associate
CEENQA
APQN
ECA
EAQAN
CHEA‐CIQG
EQANIE
Others
ReportonaScopingStudyofQualityAssuranceinHigherEducation
15
principles, none of the respondents thus far has used this external referencesystem and only some planned to do so in the near future (see Section 7:Accountabilitymechanisms).
2. QualificationsFrameworkandLearningOutcomesEstablishment of a National Qualifications Framework (NQFs) in the threeregionsplusMongoliaisatvaryinglevels.TheownershipofitsestablishmentispredominantlywiththeMinistryofEducationandSciencewith involvementofsuch stakeholders as HEIs, employers and students. In few exceptional casescountriesestablishedNQFCouncilswithinclusionofkeystakeholderstodeveloptheNQFs.Thus,theNQFsareinplacein77.3%(17countries)andendeavorsaretakentoestablishoneintherest(22.7%,5countries).However,inthecountrieswhereitdoesexistitisstillatitsinitialphaseandrobustqualityassurancemechanismsarestilltobeestablishedtoensureitsimplementation.Againstthisbackground,noneoftheNQFshasactuallybeenself‐certified–aprocedureundertakenwithinclusion of international experts to certify NQF implementation forinternational recognition purposes. In some countries pilot self‐certificationproceduresrevealedmajorgapsbetweentheprogrammeobjectivesandNQFs.AsfortheSectorialQualificationsFrameworks,45.5%oftherespondentsstatedhavingnoneand54.5%statedthatthecountryhasestablishedSQFsinsomeofthesectorsoronlyforthesectorsregulatedbythegovernment.The open‐ended questions delved deeper into the processes of developingNQFs/SQFsandstakeholderinvolvementintheprocessaswellasthemodalitiesofassessingachievementoflearningoutcomes.Themajorreflectionsregardingthedevelopmentprocesswereasfollows:
‐ establishment of committees per subject area including such stakeholders asHEIs,practitioners,ministries
‐ the QAA agency develops a proposal by referencing SQF levels to EQF andsubmitsittothegovernmentforapprovalandlegalization.
However,formostofthecountriesthedevelopmentsarestillintheprocessandtheprocesswasnotdescribed.AsfortheQAcriteriaensuringalignmentandassessmentofthealignmentoftheprogrammes with the qualifications frameworks, 76.2% of the respondentsstated they included respective points in their criteria. Thus, the agenciesactually assess the achievement of the learning outcomeswithin the academicprogrammesdefined in theNQFs/SQFs, through including it in thecriteriaandthecriterialookattheachievementthrough:
‐ evaluationofgraduatepapersandmasterthesis;‐ application of relevant questionnaires and recommendations for the experts
undertheauspicesoftheMinistry;‐ checkingtheprogrammesagainsttheitemsintheNQFs.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
16
3. TheQualityAssuranceProvider,GovernanceFrameworkandMissionMostofthegovernmentsinthethreeregionshaveadoptedrespectivelegislationto introduce external quality assurance systems. Some countries opted forestablishment of independent agencies (34.8%), 56.5% of the agencies in thesample are quasi‐autonomous national agencies, 8.7% are supranational non‐profitagenciesunder the jurisdictionof theMinistryofEducationandScience.Veryfewactasanadvisorybodytothegovernment.MostoftheagencieshaveAccreditationCommissions/Councils,whichalsoactasGoverning Boards charged with overall governance of the agency’s activities.Only invery fewcases thereare twomajordecision‐makingbodies–BoardofTrusteesresponsibleforstrategicgovernanceandtheAccreditationCommissionchargedwithamissiontotakedecisionsonaccreditation.As for the representation of the stakeholders in the decision‐taking bodies asignificantvariationinthecompositionofthegoverningbodieswasregistered.In 43.5% of the cases it includes representative of the government/ministry,tertiary education institutions (public and private) (78.3%), professionalassociations (52.2%), students (87.0%), industry/business representatives(56.5%)andinternationalexpertisrepresentedin6cases(26.1%).Withregardstothemissionoftheagenciesinthesample,theagenciesconsiderthe followingareas as of high importance to the agency: quality enhancement,accreditation,transparency,accountability,studentprotectionandinternationalcomparability.Asforthefunctions,86.9%statedconductinginstitutionaland91%programmeaccreditation. Additionally, some organizations stated conducting institutionalrecognition (13%, 3 cases), programme evaluation (47.8%) and 17.3%stateddealingwithcross‐borderprovidersasoneoftheirfunctions.
Chart3:FunctionsperformedbytheEQAAsTheoverallconcernofsomeoftheagenciesisthattheywereobligedtoconductmore than three types of procedures, which overburdens them and, actually,does not allow for a better performance at the individual procedure levels.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
17
Undoubtedly,provisionofmultipleprocedurescausesbothfinancialandhumanresource constraints for some of the agencies while at the same time alsoincreasing the administrative burden for the HEIs in various countries underreview.Asforthenatureoftheprocedures,theyarepredominantlyvoluntaryinnatureifdonebytheagenciesnotaffiliatedtotheministries.Thisisespeciallythecaseat programme level. The point might be made, that this approach is moresuitable for mature QA systems and established quality cultures. In newlydeveloping systemswith a centrally controlled legacy, amandatory nature, atleast for the first twocycles,ofprocedureswith links to fundingwouldensuresystem‐wide developments and enhancement if devised specifically to fit theneeds of the system (inmost of the systems borrowing of criteria frommoremature systems without adjusting to the local needs has had seriousconsequencesonthequalityofprovisions).Thus, a major implication is most systems are still in development and it isextremely important to set clear and robust quality assurance mechanismsleadingtoenhancementratherthanactingasburdenfortheHEIs.
4. HumanandFinancialCapacityWithregardstothecapacitytoundertakethequalityassuranceprocedures,theagencies in the sample have varying levels of staff size ranging from 2 staffmembersupto127.Thetypeofstaff involvedusually includesmanagerialandadministrativestaff.Toa lesserextent theyalsohaveresearchstaffother staffinvolvedonthecontentlevelwithexecutingEQAprocedures.Whenaskedaboutsatisfaction with the qualifications and sufficiency of the staff members, theagenciesseemtobesatisfiedwithbothbutnoneoftheagenciesstronglyagreedwiththeitems,whichsuggestsfurtherneedinhumanresourcecapacitybuilding.However,interestinglyenough,theagenciesdidnotdiscusstheneedforarobusttrainingplanfor itsQAAstaffmembers, toprepare leadersthatwouldactuallydrivetheQAsystemenhancement.Mostofthemjusthavethestaffaboutrighttoconduct the procedures; fewer of them had research staff (26%) to enable abetterunderstandingofthesystemneedsthusleadingitsenhancement.As for thequalificationsof the staffmembers, someof themhad requirementsfor at least having a BA degree, others set the level to MA degree. Yet otheragencies required MA degree and PhD for some of the positions. One of theagenciesadditionallyrequiredalicense,forGeneralAdministrationAct,whichisastateexamforobtainingalicenseforfunctioninginthefield.Workexperienceinthefieldwasalsosetasoneoftherequirementsinsomecases.Regardingthetopicofhumanresourcecapacity,welookedattheaccessoftheagencies toapoolof competentexpert to conductexternal evaluations. 77.3%responded they had access to such a pool for both national and internationalpeersand27.3%hadsuchapoolonlyfornationalpeers.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
18
With regards to the financial capacity, the predominant sources are fromgovernmental allocations (60.8%), evaluation services (60.8%) and grants anddonations(30.4%).VeryfewagenciesstatedhavingsuchincomesourcesasHEIsubscription(1case)andmembershipfees(1case)(seeChart4),mostofthemareconcernedwithunderfunding.
Chart4:EQAAfundingmechanismsThe open‐ended questions revealed that the agencies are in need of financialresources to allocate towards system‐wide research and identification of gapsanddevelopmentaltrends.Not surprisingly, the development and enhancement of EQAAs heavily dependontheavailabilityoffinancialresources,theabsenceofwhichhasapotentialtounderminetheircredibilityleadingtoseriousdamagestothesystem.
5. ExternalEvaluationProceduresWhenwelookedattheexternalqualityassurancemechanismsusedbydifferentsystems, accreditation at institutional and programme levels seems to bedominant. In the open‐ended questions there was a clear expression ofoverabundanceandoverlapofthetoolsinsomeofthesystems.
IR Institutionalrecognition PE ProgrammeevaluationCIT Confermentofinstitutionaltitle PA ProgrammeAccreditation
0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00%
Government
Membershipfee
Evaluationservices
Grantsanddonations
HEIsubscriptionfee
Other
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
SAR
Deskreviewandsite‐visit
Panelreport
Follow‐up
ReportonaScopingStudyofQualityAssuranceinHigherEducation
19
IA InstitutionalAccreditation C_LLL CertificationofLifeLongLearningI_ISO_C InstitutionalISOcertification TNE_R RecognitionofTransnationalEducationA_FQE Assessment of foreign qualifications
equivalence
Chart5:Qualityassuranceproceduresandmajorphases
Theexternal evaluation isdoneona cyclical basis ranging from5‐7years andpredominantlyevolvesaroundthefollowingmajorsteps:preparationoftheself‐assessmentreport,desk‐review,site‐visit,productionoftheexpertpanelreportand follow‐up on the recommendations. As the chart demonstrates there aredifferent approaches to application of themajor steps to different procedures.Predominantly,forinstitutionalaccreditation(IA)andprogrammeaccreditation(PA)allthemajorstepsareappliedinmostofthecases.However,withregardsto the other tools such as institutional recognition (IR), conferment ofinstitutional title (CIT), institutional ISO certification (I_ISO_C), programmeevaluation(PE),certificationoflifelonglearningmodules(C_LLL),recognitionoftransnational provisions (TNE_R), and assessment of foreign qualificationsequivalence(A‐FQE)thesemajorstepsareappliedatvaryinglevels.Asforthesite‐visit,itismandatoryin100%ofthecasesandthedurationofthesite‐visitisatvaryinglevelsrangingfrom1‐2daysin39%ofthecasesand3‐5daysin56.5%ofthecases,andmorethan5daysintherestofthecases.Generally EQAAs use external evaluation teams for their major evaluations(86.4%)and in13.6%cases the team isnot invited.With regards to thepanelthatconductsexternalevaluation,72.7%stated it is conductedby theexternalteamandEQAArepresentative,4.5%stateditisconductedonlybytheEQAAand27.3%stateditwasconductedonlybytheexternalevaluators.ThebodiesresponsibleforappointingtheexternalevaluationpanelsareEQAAs(85%), the Accreditation Commissions (20%) and to a lesser extent it is alsodonebytheGovernment/Ministry(10%).ThecompositionofthepanelbringsinavarietyofstakeholdersatavaryinglevelasdemonstratedinChart6.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
20
Chart6:Compositionofthepanel
Predominantlythestakeholdersinvolvedarestudents(70%ofthecases), localand international experts 90% and 80% respectively as well as professionalassociations (65%). Some of the panels also include employers (50%),professional practitioners (30%), HE administration (50%) and EQAA staffmembers(30%).TheEQAAstaffmembersmainlyactasprocedurecoordinators.As for the distribution of the responsibilities throughout the procedure, theEQAA procedure coordinators are mostly involved in all the steps supportingHEIandprogrammeintheirconductoftheself‐evaluation,guidingthepanelinthedesk‐review,planning thesite‐visit.Toa lesserextent theytakepart inthesite‐visitandreportproduction.
Chart7:RoleofEQAAsandexpertsintheproceduresAsthechartshows,thereisatendencyonthepartofEQAAcoordinatorstobeactive during the preparatory phase while during the site‐visit and reportproduction the external evaluators take a lead. During the desk‐review EQAAstaffhasactiveparticipationin63.64%cases.Asfortheroleoftheexperts,theyareinmostcasesdealingwithdesk‐reviewin72.73% of the cases, site‐visit in 100% of the cases and report production in95.45%of thecases.Toa lesserextent theyalsoparticipate in theplanningofthesite‐visit(59%).Thenextpoint refers to theactivities conductedby thepanelsduring the site‐visit.
0.00%20.00%40.00%60.00%80.00%100.00%
Functionnotperformed
Expertpanel
EQAA
ReportonaScopingStudyofQualityAssuranceinHigherEducation
21
Chart8:Activitiesconductedduringthesite‐visit
In practically all cases, such activities as interviews with faculty and staff(100%), reviewofdocumentaryevidence(100%), tourof thecampus facilities(95.5%),meetingwith theHEImanagement topresent thepreliminary results(95.5%),andinterviewwithstudents(90.9%)werestatedtobepartofthesite‐visit.Toalesserextentinterviewwithalumni(63.6%),observationofteachingpractices(63.6%),interviewswithexternalstakeholders(54.5%)andemployers(54.5%)werementioned. It is interestingtonote,thattherearestillcasesthatapplystudentassessmenttests(22.7%),whichwereprevalentduringthepost‐Soviet (pre‐Bologna) HE education system formation, when the functions ofaccreditationstillwereunder the jurisdictionof theMinistryofEducation.Thenature of accreditation in pre‐Bologna system evolved around mainly anoutcome‐based approachwith emphasis on quantitative data and the externalevaluatorsmainlylookedatthestudentexaminationresultsintwoconsecutiveyears to take decision on accreditation. In this case the HEI was awardedaccreditation status if about 65% of students managed to pass the minimumthreshold.Thus,afterthesitevisitoralreadyduringittheexternalevaluatorsdraftareporton the major findings and outline commendations and recommendations forfurther improvement. Once the draft report is ready the external evaluationteamsusuallyconsultwiththeHEIsunderevaluation(90.5%),students(9.5%),andprofessionalbodiesandindustry(4.8%each)priortoitsfinalization.The finaldecision takingon theprocedurerests, inmostof thecases,with theaccreditation commission/assembly/council – the decision taking bodies thatareactuallyactingasalsogoverningbodiesoftheagenciesexceptforsomecaseswhere there is a separate Accreditation Commission responsible for decisiontakingotherthanitsBoardofTrustees.Toalesserextent13.6%theexpertpaneltakes the decision. Considering most of the decision‐taking bodies havegovernment and ministry representatives in it and in some cases the latterpresideover thisbodyex‐officio, independencyof theproceduresanddecisiontakingisoneofthemajorissuesyettobesolved.Thereportsarepublishedinfull in59.1%cases,partly in18.2%casesandnotpublishedatallin22.7%cases.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
22
Last,butnotleast,animportantstepintheprocedureisthefollow‐uporganizedby the agency. Primarily, the follow up evolves around follow‐up site‐visit(63.6%), request for a remedial plan (77.3%), and request for a status report(63.6%). To a lesser extent there is an evaluation exercise conducted anew(31.8%).An open‐ended question also looked at the challenges that the agencies arefacing while conducting their external evaluation procedures. The challengesthatwereamongthemostfrequentonesareasfollows:
‐ lack of capacity on the part of the HEIs to produce an analytical self‐evaluation report. It is mainly descriptive in nature providing littlebackgroundforunderstandingthecapacityoftheHEI/programme;
‐ thenewlydevelopingcultureofapeer‐reviewintheareastillencounterslack of qualified local experts to come upwith valuable insights on theHEI/programmefunctioningand,therefore,recommendationsleadingtoenhancement;
‐ follow up procedures are not yet fully in place and are not clearlydescribed;
‐ thelanguageofthereportsproducedbytheexternalevaluatorsisstillachallenge,sinceitmakesunclearwhattherecommendationsarebasedonandwhattheyentail;
‐ under funding of those agencies prevents the latter to research therespectiveenvironments,findoutthegapsanddevelopmentaltrendsandensureproperstrategicdevelopmentofthesystem;italsopreventsfromactive participation at international levels since to be included in anetwork and participate in its activities costs money. Even more, if anexternal recognition against ESG or INQAAHE standards is to beundertaken,thentheagenciesfindthemselvesdeprivedoftheluxury;
‐ compliance with ESGs, becoming ENQA member and thus beingregistered in EQAR is yet another challenge the agencies have to fightwith. Inmostof the cases thegovernmentalarrangementsdonotallowfor this sort of compliance because of the controlling nature of thegovernments;
‐ havingstrongcandidatesasaninternationalpeerisachallengeforsomeof the agencies, since it is difficult to find those that fitted to theprocedureandcontexts;
‐ in some systems, multiplicity of various review procedures leads tosuperficialityofapproachesanddiminishedtrustinQA;
‐ inthesystemswhereQAhasbeenfordecades,thereisanurgentneedtorevisetheapproaches–criteriaandprocedures–tomeettheneeds;
‐ in somesystems, frequentchangesat thepolicy‐making level (legalandregulatory frameworks) make the mechanisms applied inconsistentlyappliedthroughoutthecases;
ReportonaScopingStudyofQualityAssuranceinHigherEducation
23
‐ lackofNQFwas alsomentionedas oneof themosturgent issues tobesolved.
‐ Assessmentofstudentlearningoutcomes‐ Involvementofstudentsinevaluationprocedures‐ Mergingofinstitutionalandprogrammeaccreditation‐ Introduction of a comprehensive system for feedback on the agency’s
activities–IQAoftheagencies;‐ The need to revise the programmes to bring in a learning‐outcome
approach.
Thus,asseenfromtheconcernsraisedbydifferentsystems,primarilycapacitybuilding of external reviewers and EQAA staff in conducting procedures,interpretingstandards,evaluatingtheHEIsandprogrammesandcomingupwithrelevantandenhancement‐drivenrecommendationsseemstobecrucialat thisstage.Anothermajorconcernrelatestobuildingonthecapacityofthesystemtoadopt a learning‐outcome approach to programme delivery and understandbetterthemethodsof learningoutcomeassessment.Last,butnotleastconcernrelates to the follow‐up. As the respondents stated, there is a need to betterunderstandrespectivemethodologiesandtheirapplicationinpracticetoensuresystemenhancement.
6. QualityAssessmentCriteria:DevelopmentandImprovementAll the countries inour sample in the three respective regions referred to twomajor sets of criteria applied in the external evaluation – national qualitystandards(91.3%)andESGs(82.6%).In8.7%ofcasesISOwasalsoappliedandin26.1%ofcasestheagenciesappliedeithercriteriadevelopedbytheagenciesthemselvesorsubjectspecificcriteriasuchasWFMEstandards.47.6%oftheagenciesapplyfieldspecificcriteriawhenconductingprogrammeaccreditation or evaluation and 52.4% apply generic criteria for the sameproceduresattheprogrammelevel.Whendevelopingandrevisingtheaccreditationcriteriaalltheagenciesstateditwas done with broad involvement of such stakeholders as students (86.4%),governments (68.2%), tertiary education institutions (90.9%), professionalassociations(63.6%)andrepresentativesofthelabormarket(77.3%).Insomecasesadditionally internationaland localexpertswere involved in theprocess.Whenaskedaboutthemeansofinvolvementtheagenciespredominantlystatedorganization of workshops and conferences. Some agencies in addition statedadministrationofsurveysandroundtablediscussions.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
24
Chart9:Involvementofstakeholdersintherevisionofprocedures
Thebodiesthattakedecisionsonthecriteriaareeitherthegovernments/ministries(43.47%),accreditationcouncils(43.47%)ortheagenciesthemselves(13%).An in‐depth analysis of the newly emerging systems has revealed that in theprocess of developing criteria in most of the cases the EQAAs draw oninternational best practice. In some cases those practices are adopted withconsiderationofthelocalneeds.However,inothercasesthesestandardsdonotfind the necessary refinement tomeet the needs of the system, thus, result indamagingratherthanimprovingit.Another major concern is related to the interpretation of standards and thecapacitytoworkwiththosestandards,whichareactuallyattheheartofsuccessofanysystem.
7. AccountabilityMechanismsWealsolookedattheaccountabilitymechanismsappliedbyvariousagencies.Itisgoodtonote,thatalmostallagencieshaveconflictofinterestpolicy, internalqualityassuranceofagenciesandappealssystemasmajoraccountabilitytoolsinplace.Whilemostagenciesarealsoreviewedbynationalauditingbodies,fewerstate compliancewithENQA standards, EQAR and INQAAHEGGP.As Chart 10showssomeagenciesareeitherintheplanningorimplementationphasegettingreadytoberecognizedagainstENQAESGsandINQAAHEGGP.Only7agencies(30.43%) have actually undergone international review against ESGs and thetermofvalidity fortheENQAmembership isalreadyexpiredforoneagencyin2012.ISOisappliedonlyinonecaseandthereisnotangibletrendonthepartofotherstooptforthataccountabilitymechanism.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
25
(IQA)internalqualityassuranceofEQAA(CoI)conflictofinterests(NR)nationalreviewbyauditingbodies(GGP)INQAAHEGuidelinesofGoodPractice
(EQAR) EuropeanQualityAssuranceRegister(ENQA) European Network of QualityAssurance(ISO) International Organization forStandardizationAppealssystem
Chart10:Accountabilitymechanismsinpractice
Only in two cases an additional accountability mechanism in the form of aNational Register of QA Agencies, which lists the agencies (local andinternational) that meet the criteria set by the given governments, wasregistered.Thus, the role of agency’s internal quality assurance mechanisms cannot beoveremphasized.ToensureeffectiveandefficientfunctioningtheEQAAsshouldpayparticularattentiontotheestablishmentandoperationalizationof internalqualityassurancesystemsattheagency level.Asforthe internationalvisibilityand credibility the EQAAs undergoing an international recognition procedureagainstcommonlyacceptedstandardsshouldbecomepriorityontheagendaoftheagenciesandrespectivegovernments.
8. StakeholderInvolvementStakeholder involvement is one of the prerequisites for establishment of aqualityculture,sinceitmakestheQAsystemlegitimate,raisesownershipofandcommitmentofthestakeholders,thusleadingtocontinuousenhancementoftheHEsysteminthecountry.The agencies in the sample are involved in the quality assurance activities atvarious levels and to varying extent. Involvement of such stakeholders asstudents, governments, employers, HEIs is expressed through participation inthedecision‐making(91.30%),developmentofproceduresandcriteria(66.7%)throughconsultation,implementationofprocedures(95.65%),capacitybuilding
0.00%10.00%20.00%30.00%40.00%50.00%60.00%70.00%80.00%90.00%100.00%
Fullyestablished
implementationinprocess
beingplanned
plannedwithoutschedule
notplannednotestablished
ReportonaScopingStudyofQualityAssuranceinHigherEducation
26
events, conferences (61.9%)workshops (66.7%). Only in 19.0% cases did theagencies mention not taking measure for involvement. Other measures ofinvolvementevolvearoundorganizationofroundtablediscussionsandsurveys.When asked about further potentials for an increased involvement ofstakeholderstheagenciespredominantlystated:
‐ revision of the existing legal frameworks to encourage such an involvement;overallrevisionofHEstrategytounderpintheroleofstakeholderinvolvementinit;
‐ increaseinfundingallowingformoreinteractionwiththestakeholdersthroughmassmedia,betterPRactivities,in‐depthresearchintothesystem‐wideneeds,deliveryofmoreworkshopsandorganizationofroundtablediscussions;
‐ frequentorganizationofcapacitybuildinganddisseminationevents;‐ making the QAA more proactive in their activities with the HEIs and other
stakeholders;‐ conveying the values and set of incentives of quality assurance to the key
stakeholders;‐ includingthestakeholdersinthedecision‐making.
9. CapacityBuildingAs for thecapacitybuilding it takesplaceat theagencystaff level (81.8%), forlocal and international evaluators (85%), IQA staff of HEIs. For the IQA of theHEIs31.8%oftheagenciesrespondedtheyorganizeditseveraltimesperyear,36.4%diditonceperyear,27.3%conductedtrainingsratherseldom,and9.1%didnotconductatall.Themostfrequenttopicsfortrainingtheexternalevaluatorsincluded
‐ approachestoevaluation;‐ legislationandrecentdevelopments;‐ interpretationofcriteriaincludingtheESGrevision;‐ qualityassurance:internalandexternal;‐ approachestoevaluationbasedonsubjectspecificcriteria;‐ goodandbadpracticestoexternalevaluation;‐ internalregulationsoftheagency.
AsfortheHEIsthetopicsevolvedaround‐ introductionofchangestolegislation;‐ introductiontoQAregulationsandprocedures;‐ internalqualityassurance:approachesandmechanisms‐ approachestoself‐assessment‐ externalqualityassurance.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
27
10. NetworkingToensureanagency is in linewiththedevelopmentsat the international levelmostly the agencies in our sample tend to be involved in one ormore qualityassurance networks. Almost all the agencies have special staff memberresponsiblefortheEQAA’sactivitieswithinthosenetworks(81%).Mostly, the agencies state the benefits of involving in the regional andinternationalnetworksas thoseofenrichingknowledgeonQAbysharingbestpractices (95.5%), and increased legitimacy and international recognition(86.4%).Asotherbenefitsenhancementof thesystemaswellaspotentials forenrichingtheirpoolsofinternationalexpertswerementioned(9%).As for the nature of participation in those networks the agencies reflected onsuchmodesasparticipationinannualmeetings,participationindevelopmentalactivities, contribution to the newsletter, organization of workshops,involvementintheboardsofthenetworksaswellasintheworkinggroups.
Chart11:NatureofinvolvementininternationalnetworksAsseenfromtheChart,theagenciesmostlyparticipateintheannualeventsandtosomeextenttheycontributetothenewslettersandregulatoryandproceduraldevelopments. Considering such networks tend to have representation of theregionscoveredontheGoverningBoardssuchkindofinvolvementalsoseemstoactivate and encourage respective countries through instilling the sense ofresponsibilityandownership.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
28
V. CONCLUSIONSCompared to other higher education systems,which enjoy a history of severaldecades (UK and Ireland) or even centuries (USA) in a systemic approach toexternal and internal quality assurance in higher education, the countries inEasternEurope,Caucasus,CentralAsiaandMongoliahavebeenendeavouringtoestablisharobustsystemofexternalandinternalqualityassuranceonlyforthelast twodecades. The countries in the target regions started to establishEQAsystemswithvaryinggovernancemodelsandextentofsuccessasearlyasatthebeginningofthe1990s.AfterjoiningtheBolognaprocessstartingfrom1999,amajor revision to EQA has been observed, underpinning such concepts asindependence,qualityenhancementand,consequently,establishmentofqualityculture.Currently,thecountriesinthesampleareinthestageofenhancingtheadopted approaches to achieve better integration at the national andinternationallevels.INQAAHE has conducted a scoping study on the state of affairs of the threeregions inNovember 2013‐March 2014. Themajor findings have revealed thefollowing developmental trends in the countries of Eastern Europe, Caucasus,CentralAsiaandMongolia.First, the initiative of establishing EQAs in the target regions came from thegovernments as opposed to thepractices in suchdecentralized systems as theUSA and New Zealand, where the initiators were the HEIs themselves. Whilecommendableonthepartofthegovernmentsandministries’inthesetupoftheEQAs at the initial stage, later on in the development process the extensiveinterference from the governments and ministries seems to impede HEIdevelopment leading to infringement of HEIs autonomy, decreasedaccountabilityand thereforequalityofprovisions. In someof thecountries theEQA is still under the jurisdiction of the Ministry of Education and suchimplementation of EQAquestions its compliancewith internationally acceptedgoodpracticesleadingtothesystemenhancement.Moreover,ontheissuesofgovernanceaswellasindependenceoftheEQAAs,theagencies are still influenced by the participation of ministry and governmentrepresentatives, who, in some of the cases, preside over the decision‐makingbodiesonanex‐officiostatus.Ontheotherhand,someofthegoverningbodiesinclude international experts, which bring in diversity and impartiality indecision‐taking tosomeextent.Thus, independenceofEQAAs’decision‐makingshould beunderpinned to ensure impartiality of decisions and increased trustandcredibilityoftheexternalqualityassurance.Second,NationalQualificationsFrameworksarealreadyestablished insomeofthe countries and are in the making in others. Some systems have alreadylaunched tools measuring and assessing alignment of study/academicprogrammeswiththeNQFs.However,theneedtodevelopandimplementtools
ReportonaScopingStudyofQualityAssuranceinHigherEducation
29
ensuring alignment of the study/academic programmes with NQFs as well ascheckingthatalignment,hasnotbeeninitiatedinmostofthecountriesyet.SuchtoolsarenecessarytoattestthevalidityandrelevanceoftheNQFs/SQFsaswellasthestudy/academicprogrammes.Third,capacitybuildingof theEQAAstaffandtheexternalevaluatorsseemstobe essential for organization of credible procedures and production of reportsthat would benefit the systems with their recommendations. Still seriouschangesaretobetakentoclosethequalityassuranceloopbyfurtherimprovingthesystemoffollow‐upontherecommendations,thus leadingtoimprovementandenhancement.Apro‐active involvementof theEQAAstaffandtheexternalevaluators in the activities of the international and regional networks mightserveasoneofthebestwaysofcapacitybuilding.Fourth,financialcapacityoftheEQAAsinthetargetcountriesisatriskbecauseof the underfunding from the government and lack of alternative revenuesources. Some systems manage to generate incomes from such alternativesources as grants and donations, and still others build on this componentthroughsettingmembershipfees.Thus,themanagementoftheagenciesneedsafurther capacity building to enable revision of approaches to funding andgenerationofalternativeincomesotherthangovernmentalallocationsandfeesfromtheevaluationprocedures.Thelatter,accordingtotherespondents,isnotenoughforundertakingresearchanddevelopmentalactivitiesbytheEQAAs.Fifth, involvement of stakeholders seems to be one of the significant featuresacrossthesystems.Theexternalreviewers’panelsincludethekeystakeholdersin most of the cases. In some cases, additionally, international peers are alsorepresented and the procedure is conducted in two languages for increasedrecognition of the procedures at the international level. However, suchmajorimpedimentsasrestrictions in the legal framework, lackof funding, scarcityofcapacitybuildingeventsanddisseminationactivitiesandtoalesserextentlackofinclusioninthedecision‐makingprocesseshavebeenidentified.Last, but not least robust accountability mechanisms for EQAAs are to beestablishedinmostofthesystemsand inthosethatarealreadyestablisheditsfurther improvement is crucial for effectiveness and efficiency of the EQAA’soperations. Such exercise as external recognition of the agency againstinternationallyacceptedstandards(ESGs,GGP,ChibaPrinciples)shouldbetakenseriouslyandregularlyiftheagenciesaretoberecognizedforcredibilityoftheiroperationsatnationalandinternationallevels.Consequently,althoughtherehavebeenseriouschangesinthehighereducationsystems allowing establishment and implementation of independent externalquality assurance systems, further investments are needed to lead to theimprovement,enhancementandincreasedcredibilityinthesystems.
ReportonaScopingStudyofQualityAssuranceinHigherEducation
30
VI. APPENDICES
AppendixA:ListofAcronymsAPQN Asia‐PacificQualityNetwork
CEENQA CentralandEasternEuropeanQualityAssurance
CHEA‐CIQG CouncilofHigherEducationAccreditation–CHEAInternational
QualityGroup
EAQAN EurasianQualityAssuranceNetwork
ECA EuropeanConsortiumofAccreditation
ENQA EuropeanNetworkofQualityAssurance
EQA ExternalQualityAssurance
EQAA ExternalQualityAssuranceAgencies
EQANIE EuropeanQualityAssuranceNetworkforInformaticsEducation
EQAR EuropeanQualityAssuranceRegister
ESG StandardsandGuidelinesforQualityAssuranceintheEuropean
HigherEducationArea
GGP GuidelinesofGoodPractice(INQAAHE)
GIQAC GlobalInitiativeonQualityAssuranceCapacity
HEI HigherEducationInstitution
INQAAHE InternationalNetworkofQualityAssuranceAgenciesinHigher
Education
IQA InternalQualityAssurance
ISO InternationalOrganizationofStandardization
NQF NationalQualificationsFramework
QA QualityAssurance
SQF SectorialQualificationsFramework
UNESCO UnitedNationsEducational,ScientificandCulturalOrganization
ReportonaScopingStudyofQualityAssuranceinHigherEducation
31
AppendixB:ListofParticipantCountriesandAgencies
Country Sentquestionnairesto Externalrecognitionstatus
Responses
1 Albania PAAHE Yes2 Bosnia and
HerzegovinaHEA YesHEAARS Yes
3 Belarus NoQA agency, the functionisdonebytheMinistry
4 Bulgaria NEAA ENQArecognized
Yes
5 Croatia ASHE ENQArecognized
Yes
6 CzechRepublic
ACCR ENQArecognized
No
7 Estonia EKKA ENQArecognized
Yes
8 Hungary HAC ENQArecognized
Yes
9 Kosovo KAA Yes10 Lithuania SKVC ENQA
recognizedYes
11 Latvia AIKNC Yes12 Montenegro Centre for Studies and
QualityAssurance
No
13 Poland UAC YesPKA ENQA
recognizedNo
14 Romania ARACIS ENQArecognized
Yes
15 Russia NCPA Intheprocess YesNAA ENQA
recognized–membership
expiredin2012
Yes
AKKORK Yes16 Slovakia AC Yes17 Slovenia SQAA Yes18 Ukraine NoQA agency, the function
isdonebytheMinistry
19 Armenia ANQA Proof‐evaluatedbyEUexperts,plannedfor
2015
Yes
20 Turkey TEPDAD Yes
ReportonaScopingStudyofQualityAssuranceinHigherEducation
32
21 Moldova NoQA agency, the functionisdonebytheMinistry
No
22 Georgia NCEQE Yes23 Azerbaijan NoQA agency, the function
isdonebytheMinistryNo
24 Mongolia NCEA Yes
25 Kyrgyzstan EDNET,theinitiativeforQAinHEwascontacted
No
26 Kazakhstan IAAR YesIQAA Yes
27 Tajikistan NoQA agency, the functionisdonebytheMinistry
No
28 Uzbekistan NoQA agency, the functionisdonebytheMinistry
No
29 Turkmenistan
NoQA agency, the functionisdonebytheMinistry
No
ReportonaScopingStudyofQualityAssuranceinHigherEducation
33
AppendixC:ContactdetailsofParticipatingQAAgencies1 Country Albania
NameofAgency: PublicAccreditationAgencyforHigherEducation
Acronym: PAAHEPhone: +35542257954E‐mail: dhuratabozo@gmail.comWeb‐site: www.aaal.edu.al/en
Nameofthepersonthatfilledinthequestionnaire:
DhurataBozo
Position: Director2 Country Armenia
NameofAgency: NationalCenterforProfessionalEducationQualityAssurance,Foundation
Acronym: ANQAPhone: +37410229145E‐mail: ruben.topchyan@anqa.amWeb‐site: www.anqa.amNameofthepersonthatfilledinthequestionnaire:
RubenTopchyan
Position: Director3 Country BosniaandHerzegovina
NameofAgency: HigherEducationAccreditationAgencyofRepublicSrpska
Acronym: HEAARSPhone: +38751922081E‐mail: m.bobrek@heaars.comWeb‐site: http://www.heaars.com/en/indexen.htmlNameofthepersonthatfilledinthequestionnaire:
MiroslavBobrek
Position: Director4 Country BosniaandHerzegovina
NameofAgency: WUSAustriaforHEAAcronym: HEAPhone: +4369914340404E‐mail: Adi.kovacevic@wus‐austria.orgWeb‐site: http://www.wus‐austria.org/Nameofthepersonthatfilledinthequestionnaire:
AdiKovacevic
Position: Director
ReportonaScopingStudyofQualityAssuranceinHigherEducation
34
5 Country Bulgaria
NameofAgency: NationalEvaluationandAccreditationAgency
Acronym: NEAAPhone: +35928077812E‐mail: mpenelova@neaa.government.bgWeb‐site: http://www.neaa.government.bg/enNameofthepersonthatfilledinthequestionnaire:
MilaPenelova
Position: SecretaryGeneral6 Country Croatia
NameofAgency: AgencyforScienceandHigherEducationAcronym: ASHEPhone: +38516274851E‐mail: ddragojevic@azvo.hrWeb‐site: https://www.azvo.hr/index.php/en/Nameofthepersonthatfilledinthequestionnaire:
DurdicaDragojevic
Position: ExpertAdvisorforInternationalCooperation7 Country Estonia
NameofAgency: EstonianHigherEducationQualityAssuranceAgency
Acronym: EKKAPhone: +3726400454E‐mail: Maiki.udam@archimedes.eeWeb‐site: http://ekka.archimedes.ee/Nameofthepersonthatfilledinthequestionnaire:
MaikiUdam
Position: DevelopmentManager8 Country Georgia
NameofAgency: NationalCenterforEducationQualityEnhancement
Acronym: NCEQEPhone: +995577153301E‐mail: n.kopaleishviili@eqe.geWeb‐site: http://eqe.ge/engNameofthepersonthatfilledinthequestionnaire:
NinoKopaleishvili
Position: HeadofInternationalRelationsDivision
ReportonaScopingStudyofQualityAssuranceinHigherEducation
35
9 Country HungaryNameofAgency: HungarianAccreditationCommitteeAcronym: HACPhone: +3613440314/121E‐mail: Rozsnyai@mab.huWeb‐site: www.mab.huNameofthepersonthatfilledinthequestionnaire:
ChristinaRozsnyai
Position: ProgrammeOfficer,ForeignAffairs10 Country Kazakhstan
NameofAgency: IndependentKazakhstanQualityAssuranceAgencyforEducation
Acronym: IQAAPhone: +77171273822E‐mail: skalanova@mail.ruWeb‐site: http://www.iqaa.kz/Nameofthepersonthatfilledinthequestionnaire:
SholpanKalanova
Position: President11 Country Kazakhstan
NameofAgency: IndependentAgencyforAccreditationandRating
Acronym: IAARPhone: +77172452402E‐mail: kanapyanov@yahoo.comWeb‐site: http://www.iaar.kz/Nameofthepersonthatfilledinthequestionnaire:
TimurKanapyanov
Position: ManagerofInternationalProjects12 Country Kosovo
NameofAgency: KosovoAccreditationAgencyAcronym: KAAPhone: +38138213087E‐mail: Ferdijezhushi2010@gmail.comWeb‐site: http://www.akreditimi‐
ks.org/new/index.php/en/Nameofthepersonthatfilledinthequestionnaire:
FerdijeZhushiEtemi
Position: PresidentoftheAccreditationBoard(NationalQualityCouncil)
ReportonaScopingStudyofQualityAssuranceinHigherEducation
36
13 Country LatviaNameofAgency: Foundation“HigherEducationQuality
EvaluationCenter”Acronym: HEQECPhone: +37129283214E‐mail: Juris.dzelme@lu.lv;aiknc@aiknc.lvWeb‐site: http://www.aiknc.lv/enNameofthepersonthatfilledinthequestionnaire:
JurisDzelme
Position: ChairmanoftheBoard14 Country Lithuania
NameofAgency: CenterforQualityAssessmentinHEAcronym: SKVCPhone: +37061235339E‐mail: Nora.skaburskiene@skvc.ltWeb‐site: http://www.skvc.lt/enNameofthepersonthatfilledinthequestionnaire:
NoraSkaburskiene
Position: ActingDirector15 Country Mongolia
NameofAgency: MongolianNationalCouncilforEducationAccreditation
Acronym: MNCEAPhone: +97670109391E‐mail: tungalag@accmon.mnWeb‐site: www.accmon.mn/enNameofthepersonthatfilledinthequestionnaire:
JamtsTungalag
Position: SeniorOfficerforInstitutionalAccreditation16 Country Poland
NameofAgency: UniversityAccreditationCommissionAcronym: UACPhone: +48510249543E‐mail: palka@amu.edu.plWeb‐site: http://www.uka.amu.edu.pl/english.phpNameofthepersonthatfilledinthequestionnaire:
SbigniewPalka
Position: Professor
ReportonaScopingStudyofQualityAssuranceinHigherEducation
37
17 Country RomaniaNameofAgency: RomanianAgencyforQualityAssurancein
HigherEducationAcronym: ARACISPhone: +40758075858E‐mail: Mihai.floroiu@aracis.roWeb‐site: http://www.aracis.ro/nc/en/aracis/Nameofthepersonthatfilledinthequestionnaire:
MihaiFloroiu
Position: HeadofInternationalRelationsService18 Country RussianFederation
NameofAgency: AgencyforQualityAssuranceinHigherEducationandCareerDevelopment
Acronym: AKKORKPhone: +79037236467E‐mail: Erika.soboleva@gmail.comWeb‐site: http://www.akkork.ru/e/Nameofthepersonthatfilledinthequestionnaire:
ErikaSoboleva
Position: HeadofInternationalRelations19 Country RussianFederation
NameofAgency: NationalAccreditationAgencyAcronym: NAAPhone: +74951346886E‐mail: imo@nica.ruWeb‐site: www.nica.ruNameofthepersonthatfilledinthequestionnaire:
TatianaBarhatova
Position: Director20 Country RussianFederation
NameofAgency: NationalCenterforPublicAccreditationAcronym: NCPAPhone: +78362421640E‐mail: Galina_motova@mail.ruWeb‐site: www.ncpa.ruNameofthepersonthatfilledinthequestionnaire:
GalinaMotova
Position: DeputyDirector
ReportonaScopingStudyofQualityAssuranceinHigherEducation
38
21 Country SlovakiaNameofAgency: AccreditationCommission,AdvisoryBodyof
theGovernmentoftheSlovakRepublicAcronym: ACPhone: +421259374325E‐mail: Akredkom2@minedu.skWeb‐site: http://www.akredkom.sk/en/Nameofthepersonthatfilledinthequestionnaire:
LuborFisera
Position: Chairman22 Country Slovenia
NameofAgency: SlovenianQualityAssuranceAgencyAcronym: NAKVISPhone: +38640636923E‐mail: Ivan.leban@nakvis.siWeb‐site: http://www.nakvis.siNameofthepersonthatfilledinthequestionnaire:
IvanLeban
Position: Director23 Country Turkey
NameofAgency: AssociationforEvaluationandAccreditationofMedicalEducationPrograms
Acronym: TEPDADPhone: +905323632634E‐mail: durakhi@gmail.comWeb‐site:Nameofthepersonthatfilledinthequestionnaire:
H.IbrahimDurak
Position: SecretaryGeneral
ReportonaScopingStudyofQualityAssuranceinHigherEducation
39
INQAAHESecretariatCatalanUniversityQualityAssuranceAgency,AQUCatalunya
C.delsVergós,36‐42,08017BarcelonaTel:+34932688950
e‐mail:secretariat@inqaahe.org
Recommended