Briefing on “Towards a Quality Assurance

Framework for Education & Training Boards:

Quality Assurance Guidelines”

Phase 1Task Responsibility Timeline

1 Develop Draft QA Guidelines QQI Complete

2 Draft Guidelines agreed for consultation with ETBI/FET Forum

QQI Complete

3 External Advisors to conduct feasibility study on draft guidelies

Joint QQI/ETB Jan 2015

4 Report on feasibility of draft guidelines QQI Feb 2015

5 Revisions of Draft Guidelines following consideration within ETBs and QQI

QQI Feb 2015

6 QA Guidelines for ETB Sector established QQI Mar 2015

Phase 2Task Responsibility Timeline

7 ETBs use Guidelines to conduct self evaluation ETB TBD

8 Each ETB publishes Self Evaluation report ETB TBD

9 External Panels conduct Reviews using ETBs Self Evaluation Reports. - Visits, Reports and Recommendations

QQI TBD

10 Each ETB produces an Improvement Plan to address External Panel Report

ETB TBD

11 Future engagement with each ETB based on that ETB's Improvement Plan

QQI TBD

12 System Enhancement Programme of corporate and programme level quality assurance processes.

ETB TBD

13 Delegation of Authority to ETBs

Feasibility Study• QQI testing draft guidelines to ensure that they are feasible and

appropriate to the current operating context of the ETBs. • A small team of external advisors to test the guidelines by

engaging with the ETB sector. • A mechanism of consultation that will ensure any significant gaps

in the guidelines are captured. • The advisory team testing the feasibility of the guidelines in

three different ETBs collectively representative of the diversity of the sector.

• The outcome of this stage will be the feasibility study report.

What is the Feasibility Study about?

Short scoping process to test the feasibility of draft QA guidelines to enable QQI to: • ensure the guidelines are appropriate to the new structures of the ETB

organisations• understand the broader operating context of the ETBs• ensure the guidelines are fit for purpose for the ETBs; taking on board the full

operating context including relationship with QQI and other ETB stakeholders• facilitate a holistic approach for development of the final version of the guidelines

and the development of a QA system in each ETB.• identify any significant gaps in the guidelines • consult with the full range of ETBs on the draft guidelines and provide each one

with an opportunity to give feedback • avail of some expertise and experience from FE systems in other jurisdictions with

the aid of the independent advisors• ensure the QA guidelines and criteria and fit for purpose in light of QQI’s remit

ScheduleJanuary 2015 : ETB QA Guidelines Feasibility Study Schedule

Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3

4 5 6 7 8 9 10

11 12 13 14 15 16 17

18

19 Panel Meeting QQI 12-3pm

20 21 22 23 24

25

26 External Panel Meeting [Limerick?]

27 External Panel: LCETB [Limerick]

28 External Panel: LMETB [Dundalk]

29 External Panel: CMETB:[Cavan] QQI Panel: Cork ETB

30 31

ScheduleFebruary 2015

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

1 2 QQI Panel: CDETB

3 QQI Panel: LOETB [Tullamore]

4

5 6 7

8 9 10 11 QQI Panel: GRETB [Galway}

12 13 14

15 16

17 18 19 20

21

22

23

24 25 26 27 28

Panel Members

Mr. Brian Lister• Retired (Educational Consultant)• Principal and CEO of Stevenson College Edinburgh• Chair of the Board of Directors at Newbattle Abbey College Residential FE College• Principal and CEO at Cumbernauld College near Glasgow.• CEO at the Scottish Further Education Unit, SFEU• OBE for services to Further Education in 2008.

Mr. Gerard Looney• Education Consultant and Mediator.• Deputy Principal of Colaiste Stiofain Naofa, Cork• Principal of St John’s Central College, Cork• Former President of the National Association of Principals and Deputies (NAPD)

Panel Members

Mr. Maurice Doran • Former Registrar of IT Sligo. • He was the first Registrar of the Institute from 1993 to 2001 developing the

Institute’s academic quality systems and awards. • He had extensive industry experience in the heavy steel and iron industry abroad

before joining IT Sligo as Head of the Engineering School in 1970.• Since 2001, Maurice has been on secondment to the DES and the Higher Education

Authority (HEA), to advise on policy relating to apprenticeship; culminating in the Apprenticeship Review established by the Minister in 2013.

Panel Visit – Tuesday 27th January10am-1.00pm Meeting with ETB Management Staff [Interactive Presentation]

1. Overview and Introductions [Panel Chair] 2. Overview of the ETB [ETB Management]

Structure and ProfileUpdate on current internal QA arrangementsStrategic and internal and external changes impacting on quality assurance, including QA amendments/interim QA arrangements since formation of ETBQuality assurance developments and priority areas for 2015

3. Feedback to Advisory Panel on the Draft QA Guidelines for the ETBs 2.00pm-4.00pm Meetings with other staff and Feedback 1. Panel meetings with staff [profile and roles] 2. Discussion on draft QA guidelines

3. Summary feedback and Close

Preparation

• Provide materials/data• Document list being prepared on QQI identified topics including -

Strategy and Planning, Organisation Structure, Programmes, QA System(s), Internal Monitoring System, Collaborative Relationships, Solas / DE&S Reporting requirements, Management Information Systems, and Learner Involvement.

• FET SMT Attendance at AM Panel Meeting• LCETB FET SMT attendance, 10am to 1pm, January 27th, Boardroom,

Cecil Street.

• Nominate staff to attend PM Panel Meeting• Programme representatives required – ALO, Programme co-ordinator,

TSO etc. 2pm to 4pm.

• Feedback on Guidelines• Feedback on guidelines to be prepared in advance of visit.

When What WhoFriday 16th Jan Update of LCETB 2012 Audit

Preparation of Material Pack for PanelTara and Alan

14th January QQI QA Update with TSO’s LCETB Programme Development and Validation Policy QA Reengagement and Feasibility Study

Tara and AlanTraining Centre Mgrs,TSO’s

15th January Briefing for Senior Management Team Overview of Visit, Proposed Plan, Summary of Guidelines and questions

Tara and AlanFET SMT

15th January Guidelines and Questions circulated for feedback to selected staff Alan

16th January Deadline for Pack to be sent to QQI Tara and Alan20th January QQI QA Update with PLC’s Principals and QQI Co-ordinators

LCETB Programme Development and Validation Policy QA Reengagement and Feasibility Study

Tara and AlanPLC PrincipalsQQI Co-ordinators

23rd January Presentation Finalised and circulated to FET SMT for agreement Tara and Alan

TBC Briefing with CEO Alan

27th January External Panel Meeting with FET SMT Structure and Profile of LCETB Current internal QA arrangements Strategic and internal and external changes impacting on QA Quality assurance developments and priority areas for 2015 FET SMT Feedback

FET SMT, Panel, CEO of LCETB

27th January External Panel Meeting with Staff Roles and Profile Discussion on draft QA guidelines Summary feedback and Close

Development Team Selected ManagersTSO’s and QQI Co-ordinators

LCETB FET Service-QA Guidelines Feasibility Study Plan

Consultative Questions Agreed1. Are the guidelines sufficient to help the ETBs to develop their own procedures in the

following policy areas?Policy areas listed 8.1 – 8.11If not, please comment

2. Are there other areas which should be included?

If so, please list, 3. Will the guidelines contribute to the enhancement and development of an ETB’s

quality assurance? If no, please comment

4. Overall, is the language clear and appropriate?

If no, please give examples 5. What are the challenges for your ETB/the FET sector in implementing the guidelines to

develop procedures and processes?

Draft Guidelines Overview• The key message here is that Quality Assurance is at corporate level. The

guidelines promote the establishment of higher order quality assurance processes. New areas/processes are as follows:– Governance structures required to assure oversight of policy development and

implementation. – Policy development to ensure transparency and consistency across the spread of

the ETB provision.– Sub-contracting/procuring programme delivery – linked to collaborative provision:

to ensure that for any ETB programme(s) wholly or partly delivered by another provider, there is a formal agreement setting out each party’s responsibilities.

– Information management system required– Monitoring to assure that governance structures have adequate information to

identify areas of potential concern and areas of excellence – link to Information Management System

• Complementary Corporate and Centre Level responsibilities

Purpose of Guidelines• These guidelines are intended to assist ETB providers in the

development of appropriate quality assurance systems which will promote and support quality in ETB programmes and services.

• The guidelines do not prescribe how ETBs are to carry out their work but will specify the areas of provision which QQI considers crucial to quality and for which ETBs are expected to establish and implement policies and procedures.

• It is essential that these guidelines be understood as an aid to the establishment of an ETB quality system rather than as a ‘how to’ manual. The guidelines will assist an ETB to:– provide transparency of policy and process – establish systems to monitor and review effectiveness– facilitate enhancement of programmes and services

Principles

• The primary responsibility for quality and its assurance lies with providers

• Quality assurance requires planning, resources and commitment• Continuous improvement must be the goal of a quality

assurance process• Quality systems should enhance transparency • Quality systems are context dependent i.e. the scale and scope

of a provider’s provision will impact on how it operates quality assurance

• Ownership and understanding of a provider quality system by its management and staff are crucial.

Policies Vs Procedures• Policies - A policy is a documented statement of the ETB principles and

approach to a particular area of education / training. It should be consistent with the overall Mission and should provide an underpinning rationale for all staff at corporate and/or centre level working in that particular area.

• Procedures - To be translated into practice(s), a policy must be broken down into one or more clear and coherent procedures. These are statements of how the processes are to be carried out and their development will be informed by the relevant policy.

• Given the layered and distributed nature of an ETB, it would be expected that there will be different procedures required in different contexts to collectively comply with policy in that area.

8.1 - Governance

• An ETB should have a system overseeing the work of the entire organisation to ensure its validity. This system, often called a governance structure, should enforce separation of responsibilities between those who produce / develop material(s) and those who approve it.

• Included in the structure should be groups which approve:– draft policies – draft programmes prior to submission for validation– self-evaluation and improvement reports– annual indicator reports

• These groups should be identified in the provider’s organisation chart and their terms of reference should be documented.

8.2 – Quality Management• It is crucial … to have a commitment to quality and a system to translate the

commitment into reality. Such a system will demonstrate that a provider understands what it means to deliver high quality programmes and services in a regulated context and that the understanding is disseminated throughout the organisation.

• Description of processes • Monitoring of processes – indicators• Self-Evaluation and Improvement• Governance structures

Quality Policy

Organisational Structure Governance Structures

Management Responsibility Information Systems

Designated Responsibility

8.3 - Staff Recruitment & Development

It is essential that every ETB has a systematic approach to the recruitment, allocation and further professional development of people engaged in programme and service delivery.

• ETB policy on Staff Recruitment and Development should express its commitment to appoint suitably qualified staff to the role of teacher / instructor and to provide opportunities for further development.

The associated procedures should address the following areas:Staff Recruitment and Allocation: • The general criteria and approach used in the appointment of programme staff should be

clearly stated. Where necessary, more detailed selection criteria should be used. Staff Development:• On-going development of staff capacity is crucial to programme quality. A mechanism needs

to be in place to impart feedback to staff members on their strengths and areas requiring improvement.

• Planning and resources need to be committed to identifying and addressing staff training needs.

8.4 – Teaching and LearningETB Teaching and Learning Policy The policy should state the ETB’s commitment to monitoring and improving the quality of teaching and learning on its programmes. It should have associated procedures which clarify how the policy will be implemented and should address the following areas:• Programme Profile: • All programme designs should identify:

– teaching and skills expertise required for the programme – modes of teaching and learning appropriate to the programme context and

objectives.• Feedback on Programme Experience

– Feedback should be sought from all main stakeholders i.e. staff, learners, HEIs / Employers, past learners on their experience of the programme and their suggestions for programme improvement.

– Feedback should be sought on issues such as:• Programme content, Delivery modes, Learning resources (IT facilities, reading

materials etc), Assessment, Information, Achievement of objectives, Support services, Accommodation for diversity

8.5 – Access, Transfer and Progression

• The Qualifications and Quality Assurance (Education and Training) Act (2012) sets out responsibilities for QQI and for providers in the areas of Access, Transfer and Progression. Provider responsibilities remain more or less as they were prior to this legislation, increased emphases in some areas, - Information for Learners.

• The ETB’s Access, Transfer and Progression policy should demonstrate its awareness of its obligations arising from the legislation

8.5.1 – Information for Learners (detailed specifics from 2012 Act)8.5.2 – Leaner Entry Arrangements (level of English & language supports available)8.5.3 – Recognition of Prior Learning (develop a statement of the arrangements provided, if any, in respect of the RPL for entry to programmes, for credit towards an award and for access to an award) 8.5.4 – Facilitation of Diversity (“ability to participate successfully in a programme without the hindrance of unnecessary barriers”)8.5.5 – Transfer and Progression (Governance function – “In designing programmes, ETBs should seek, where possible, to offer awards which present learners with the opportunity to transfer or progress should they wish to do so, either immediately on attainment of the award or at a later date.”)

8.6 - Programme Design, Development, Delivery, Approval and Review

An ETB’s Programme Design, Development, Delivery, Approval and Review Policy should inform the work of those staff engaged in the design and delivery of programmes, so that programmes are constructed and delivered to allow learners to achieve the learning outcomes required for a specified NFQ award(s).

8.6.1 Need Identification: Governance – Programme Committee8.6.2 Programme Design: “Capacity to Succeed” statement8.6.3 Provision and maintenance of learning facilities / resources: Governance – Programme Committee8.6.4 Programme Approval: Governance – Programme Committee8.6.5 Programme Delivery: see “Teaching and Learning” section8.6.6 Learner Records: Information Management System8.6.7 Premises: B.5.8 Health & Safety8.6.8 Programme Review – Revalidation:self-evaluation and review as part of the re-validation process.

8.7 – Fair and Consistent Assessment of Learners

ETB policy on Assessment of Learners in conjunction with 2013 revised “Quality Assuring Assessment, Guidelines for Providers”8.7.1 - Information to Learners: as previous8.7.2 - Coordinated Planning of Assessment: as previous8.7.3 - Security of assessment related processes and material: records only8.7.4 - Reasonable Accommodation: to include ESOL8.7.5 - Consistency of marking between assessors: monitoring consistency / remedial actions8.7.6 - Workplace Assessment: be briefed/trained in order to conduct the assessment.8.7.7 - Assessment of Distance / e-learning based programmes: ensure validity and consistency of assessment8.7.8 - Internal Verification: Ensuring consistency and validity of assessment is now part of this process8.7.9 - External Authentication: as previous8.7.10 - Results Approval: 2 layers, centre (as previous and including comparative grade analysis)and governance (review results data on a periodic basis to ensure consistency across centres and between programmes)8.7.11 - Feedback to Learners: as previous8.7.12 - Learner Appeals: information as part of programme information to learners

8.8 – Protection for Enrolled Learners

• The 2012 Act sets out (in Part 6) legal requirements for QQI and providers of education and training (providers) regarding the Protection of Enrolled Learners (PEL). It establishes that it is the responsibility of providers to ensure that there are adequate arrangements in place for PEL on specified programmes. QQI has published ‘Protection of Enrolled Learners: Protocols for the Implementation of Part 6 of the 2012 Act’.

• ETBs will have to provide learners with the ETB arrangements around PEL, whatever that information is.

8.9 – Collaborative ProvisionAn ETB’s policy on collaborative provision should set out any possible rationale it would have for establishing a collaborative arrangement for delivery of a programme. Any such rationale should be for the benefit of learners. The policy should also clarify the ETB’s responsibilities to its learners when its programme(s) are being wholly or partly delivered / assessed by a second provider.

8.9.1 – Collaboration AgreementA written statement detailing respective responsibilities as agreed between

the first and second providers is required and will form part of the application for validation. 8.9.2 – Monitoring Arrangements

As the first provider has overall responsibility for the quality of the programme, it is required to demonstrate how it will monitor those parts of the programme delivered by the second provider. Monitoring arrangements must be stated in the Collaboration Agreement

8.10 – Learner Record System• An ETB should have a robust and comprehensive database system capable of

– maintaining secure learner records for current use and historical review– providing reports required for internal quality management (ref 8.2)– generating data required for and compatible with QQI’s certification system (ref 8.7)– generating statistical and other reports to meet internal and external information

requirements• The ETB and its centres should be aware of and comply with, relevant

obligations under data protection legislation. These will include establishing data access controls, data backup systems and learner information material making clear what personal data will be required to be collected and for what purpose.

• Administrative and IT resources should be sufficient to ensure that the database is maintained securely and that data, particularly that relating to learner assessment, are accurate and complete.

8.11 - Self-Evaluation and Improvement of Programmes and Services

• The self-evaluation by an ETB of its programmes and services is a fundamental part of its quality assurance system. It is a way of developing through constructive questioning leading to positive recommendations and improvement planning. It should be viewed as a process primarily for the benefit of current and future learners and the provider staff.

• Self- evaluation of programmes for the purpose of re-validation (explicitly linked to re-validation)

• Self- evaluation of the effectiveness of the QA system (at least once every 7 years)

• Two different self-evaluation reports, one based on the QA systems and one linked to the programmes.

• “To provide independent confirmation or otherwise of the SER, it is normal to follow up a self-evaluation with a visit by an external panel with expertise relevant to the subject of evaluation.”

Re-Engagement with QQICompletion of the process of re-engagement will require an ETB to:

– Conduct a self-evaluation of its current quality assurance system against these guidelines

– Document the findings in a self-evaluation report (SER)– Organise an external panel to review its institutional quality assurance using the SER

and supporting documentation as well as through meeting, staff, learners and other stakeholders.

– Using the report of the external panel, develop and publish an improvement strategy document, to include an action plan to address, within an agreed timescale, the recommendations of the external panel..

• This Action Plan will be the basis for future engagement between QQI and the

ETB. Over time, as evaluations demonstrate consistent effectiveness of institutional quality assurance, ETBs may consider applying to QQI for delegation of authority to validate their own programmes and make their own awards.