The Role of Accreditation and Assessment Commissions in HE QA Richard Lewis Former Pro-vice-chancellor of the Open University (UK) Immediate Past President

Role of QA Comm KSU Feb 2011 1

The Role of Accreditation and Assessment Commissions in HE QA

Richard Lewis

Former Pro-vice-chancellor of the Open University (UK)

Immediate Past President of the International Network of Quality Assurance Agencies in

Higher Education (INQAAHE)


Summary of Presentation

• The role of QA agencies• To whom they are accountable• Their remit• The “standard model” and variations thereon• Current developments including increased emphasis

on quality enhancement and the need to cater for a maturing yet diversifying HE system

• INQAAHE’s Guidelines of Good Practice for QA Agencies


Definitions

Quality AssuranceQuality assurance is an all-embracing term covering all the policies, processes, and actions through which the quality of higher education is maintained and developed. (Campbell and Rozsnyai, 2002)

Quality AssessmentQuality Assessment covers both the means by which a judgement is made about the quality and standards of an institution or a programme and the judgement itself. (Vlãscanu, Grünberg and Pârlea, 2004)

AccreditationAccreditation is a form of quality assessment where the outcome is a binary (yes/no) decision that usually involves the granting of special status to an institution or programme (CHEA 2001).


The prime functions of QA (including accreditation) Agencies

Are the institutions that fall within its remit• Doing the right thing• And are doing it right.

Which begs the questions• Who decides what is the right thing?• How does an institution demonstrate that it is

doing it right?


To whom are QA agencies accountable?

The wider community – not necessarily the same thing as government.

QA agencies should work closely with the institutions for whom they have responsibility and should remain aware that it is the institutions that have ultimate responsibility for the quality and standards of the programmes they deliver and the overall academic environment they provide for their students.


Remit of QA agencies

Programmes only 20%Institutions only 16%Both 64%

Notes• 2008 figures based on the INQAAHE database.• Above exclude programme specific agencies such as professional bodies


The “Standard” model

In an INQAAHE survey virtually all agencies stated that used the following approach • Sets of regulations and guidelines formulated• A self evaluation prepared by the institution• The appointment of a peer group whose review of the

institution or programme would start with a review of the self evaluation

• Site visits by the peer group.• The publication of a report or, in some cases, only the

decision


Different approaches to applying the basic model

Hard or Soft

Hard• Agency closely specifies content of self study

report.• Agency issues a grade as a result of the process• Agency emphasises checking minimum

standards as opposed to encouraging enhancement

• Assess rather than audit


The difference between audit and assessment

AuditQAA states that one of three objectives of institutional audit is to ensure that the institution has effective means of ensuring that the awards and qualifications in HE are of an academic standard at least consistent with those referred to in The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ) ….


The difference between audit and assessment 2

Assessment“An institutional accrediting body evaluates an entire organization and accredits as a whole. It assesses formal educational activities and also evaluates governance and administration, financial stability, admissions and student personnel services, resources, student academic achievement, organizational effectiveness, and relationships with outside constituencies”. Higher Education Corporation


Publication – contrasting approaches

Europe

“Reports should be published and should be written in a way which is clear and readily accessible to its intended readership”

(ENQA 2005)

The United States “In most cases, the Commission will not make reports public

without the permission of the college or university.”(HEC 2003)


Publication of Reports

Publishes Reports

Does not Publish Reports

Unknown TOTALS

Europe 17 11 28 Asia and Australasia

4 11 15

Africa and the Middle East

4 3 1 8

North America 3 8 11 South and Central America and the Caribbean

3 2 5

TOTALS 31 35 1 67 % ages 46% 52% 1%


In the early days QA agencies relied heavily on two main elements

• The intuitive judgement of the academic reviewers based, not on agreed explicit requirements, but on their experience

• Relatively crude quantitative input measures such as the ratio of academic staff (faculty) to students and the number of books in the library.


But things are changing

Amongst the more important emerging trends are:• A switch in emphasise from quality assurance for

accountability to quality assurance for enhancement• The change in focus from inputs to outputs• The move away from the reliance on the intuitive

experience of reviewers to an approach based on explicit statements of the requirements imposed on institutions and programmes

• Increasing attention to international issues including Cross-border Higher Education (CBHE)


Accountability and Enhancement

Accountability is concerned with the institution being able to demonstrate that it is operating at or above the basic minimum standard, while quality enhancement is concerned with the continuous process of quality improvement.


As QA systems mature

QA agencies will need to adapt their approaches after, say, two or three rounds of review or accreditation.But they might then have to deal with a range of institutions from the very well to the very newly established institutions.There are some good examples of dealing with this problem – but not very many!


INQAAHE – Guidelines of Good Practice for QA agencies

• Established in 1991 with about 10 members, now has over 200 members.

• Has for some years published Guidelines of Good Practice in Quality Assurance; latest edition August 2007. Specifically addressed to External Quality Assurance and Accreditation Agencies (EQAAs).

• Consistent with other guidelines; notably those of the ENQA (European Network of EQA bodies)

• Been adopted by many agencies as a yardstick against which to judge their own policies and procedures

www.inqaahe.org


The Guidelines - 1

1. Governance.Ownership and governance structure should be appropriate for the objectives of the agencyPractical Implications – relationship with government, independence of EQAA especially in the making of “academic judgement.”

2. Resources- Should have adequate human and financial resources for day to day operations and for developmentPractical implications – how funded?


The Guidelines - 2

3. QA of the EQAANeeds a continuous system of QA for own activities and should be subject to external review at regular intervalsPractical implications – how best to do?

4. Reporting Public InformationDiscloses its criteria etc and its decisionsPractical implications – should full reports of evaluations be made publicly available?


The Guidelines - 3

5. The relationship between EQAAs and Higher Education Institutions (HEIs)EQAAs (and others) should recognise that quality is the responsibility of the HEI and that EQAA should contribute to quality enhancement as well as accountability.Practical implication – how best deal with system where HEIs exhibit very different degrees of maturity?


The Guidelines -4

6 The EQAA’s requirements for institutional/programme performanceThe EQAA should make clear of its expectations (sometimes described as standards, factors, precepts) which should address all areas that fall within the EQAAs scope including teaching, learning, research, community work etc and necessary resources. May also include specific learning goals Practical implications – how specific should these be, how are they established and how should specific learning goals be specified?


The Guidelines -5

7. The EQAA’s requirements re Institutional Self-Evaluation

- EQAA needs to explain to institutions (and others) the self-evaluation process

- Should apply standards and criteria that have been subject to reasonable consultation with stakeholders

- The process should contribute to quality improvement.

Practical implication – how balance the interests of different stakeholders?


The Guidelines -6

8 The EQAA’s Evaluation of the institution and/or programme- EQAA should publish standards used, assessment methods, and decision criteria. Should specify approaches to selection and training of reviewers. Should ensure institutions are evaluated in an equivalent way. Where practicable panels should include one reviewer from another jurisdiction.Practical implications – include how best ensure equivalence


The Guidelines -7

9 DecisionsAn EQAA must be independent i.e. has autonomous responsibility for its operations and its judgements should not be influenced by third parties. When the EQAA advises governments or other public bodies the decisions of each agency should be made as independently as practicablePractical implications – how best to ensure independence of decision making (how best balance with needs of government). Is it realistic to expect that the EQAA can have complete autonomy about its style of operation?


The Guidelines - 8

10 AppealsThe EQAA should have appropriate methods and policies for appeals. Appeals should be conducted by those not responsible for the original decision but appeals should not necessarily be conducted outside the EQAA.Practical implications – should there be an external court of appeal, how best to avoid a situation where there are many appeals?


Summary of the GGP

Funders-Take their money but not their instructions. But talk to them as well as to other stakeholders.Do unto yourself that you do unto others-Establish effective systems of internal and external quality assurance for the agencyThe public (broader community)-Speak to it.


My thanks for your attention.I very much look forward to the discussion.

Richard [email protected]


References

Campbell and Rozsnyai, 2002, Quality Assurance and the Development of Course Programmes. Papers on Higher Education Regional University Network on Governance and Management of Higher Education in South East Europe Bucharest, UNESCO.

CHEA, 2001, CHEA web site, http://www.chea.org section on “International Quality Review”, CHEA glossary of terms

ENQA 2005, Standards and Guidelines for Quality Assurance in the European Higher Education Area, European Association for Quality assurance in Higher Education, Helsinki

HEC 2003, Handbook of Accreditation, Higher Education Commission, Chicago

Vlãsceanu, L., Grünberg, L., and Pârlea, D., 2004, Quality Assurance and Accreditation: A Glossary of Basic Terms and Definitions (Bucharest, UNESCO-CEPES)

Documents

The Role of Accreditation and Assessment Commissions in HE QA Richard Lewis Former Pro-vice-chancellor of the Open University (UK) Immediate Past President