Accreditation from the perspective of three European ... Conference/Castano... · Nov 2010 review prelim. decision and prepare response . Preparation / Workplan (1) ... Faculty for

Accreditation from the perspective of three European Higher Education

Institutions – Benefits, Pitfalls, Recommendations

Dipl.-Inform. Stefan Schiffer (RWTH Aachen University), Prof. Silvana Castano (Università degli Studi di Milano),

Prof. Janis Grundspenkis (Riga Technical University)

Como, 22/23 September 2011

Background

RWTH Aachen University established 1870 with a focus on Natural Sciences + Eng. 9 faculties (incl. medicine), ca. 32.000 students, ca. 100 study programmes 19% international students, 2.700 Graduates p.a. ca. 450 Professors, ca. 2.000 Research Assistants, ca. 2.000 non-tech. staff, ca. 1.700 3rd-party funded employees 520 Mio.EUR university budget (190 Mio. third-party funds)

University of Bonn + B-IT Bonn-Aachen International Center for Information Technology Cooperation with University of Bonn and Fraunhofer

Degrees submitted for Review (1) “Meta”-Accreditation via ASIIN, three programmes Accreditation via ASIIN (nat’l accred. agency) Additionally: EQANIE reviewed ASIIN procedure Joint accreditation of three programmes: SSE/MI/LSI Coordination from RWTH Aachen University

SSE – Software Systems Engineering (Accreditation)

MI – Media Informatics (Re-Accreditation)

LSI – Life-Science Informatics (Re-Accreditation)

Degrees submitted for Review (2) “Meta”-Accreditation via ASIIN, three programmes

SSE – Software Systems Engineering (Accreditation) RWTH Aachen University, Computer Science Department Existed as a Test Study Programme since 2001 Revised structure in starting from 2010 Focus on Software Engineering & Management Two year (120 ECTS) programme, pure English coursework





MI – Media Informatics (Re-Accreditation) RWTH Aachen Univ., Univ. of Bonn, Fraunhofer FIT+IAIS Since 2005, focus on communication or media orientation In 2009: 40 beginners, 32 graduations, generally 80-90% success rate, median study duration below 5 semesters Very good placement (60% industry, 40% science)





LSI – Life-Science Informatics (Re-Accreditation) Univ. of Bonn, RWTH Aachen Univ., Fraunhofer SCAI+FIT Since 2005, includes bio-, medical- and chemo-informatics In 2009: 20 beginners (128 applicants), 11 completed top placements (e.g. Oxford, Cambridge, Stanford, Bayer) 70% research+doctoral studies, 30% industry

• Date of Request for accreditation: late 2009 • Milestones of preparation

Oct 2009 collection of material from existing programmes

Nov 2009 co-ordinate with all parties involved

Dec 2009 trigger collection of information from faculty + profs

Jan 2010 get up-to-date information from central adm.

Feb 2010 preparation of final draft version

Mar/Apr 2010 refine parts that did not make it to draft

May 2010 Integration of Feedback on Draft Version

Sep/Oct 2010 prepare audit

Nov 2010 review prelim. decision and prepare response

Preparation / Workplan (1)

• Date Submission of draft self-assessment report 12.03.2010 submission of draft version to ASIIN

04.05.2010 feedback from ASIIN on draft version

• Date Submission of final version 19.05.2010 supplied all documents to ASIIN (7 print + electr.)

• Date of auditing visit 20.10.2010 one day audit, ASIIN + EQANIE auditors

(initial date not possible due to date conflicts)


• Date of decision(s) 19.11.2010 Preliminary Decision from ASIIN

30.11.2010 response statement from RWTH/Uni.Bonn/B-IT/... on preliminary decision

01.12.2010 Meeting of the ASIIN “Fachausschuss Informatik” with review of response statement - Internal decision making with proposal for AAC

09.12.2010 Meeting of ASIIN Accreditation Commission (AAC)

13.12.2010 Advance information on ASIIN decision

20.12.2010 Final ASIIN decision notice (with certificate and requirements + recommendation)

13.04.2011 advance notice of EQANIE decision to award ASIIN with Euro-Inf-Label concession rights


Self-Assessment Report (1)

Stakeholders involved in preparing the report Central administration of each university

Faculty for Math, Physics and Computer Science

Computer Science Department

Programme coordinators

Study advisors

RWTH Aachen International Office


Extent / approx. number of pages Report SSE: 33 pages self-assessment report MI/LSI with similar dimensions each

Extent / approx. number of pages Annexes SSE:

170 pages modulehandbook

43 pages personal handbook

23 pages examination regulations

4 pages diploma supplement

20 pages RWTH evaluation regulations

28 pages RWTH gender + diversity mgmt


3 biggest challenges in report writing?

Coordination of various different parties

Ensure that all parties involved meet the deadlines

Select important information and get up-to-date data

Benefits of the accreditation experience

Valuable external feedback from auditors Proportionally low extra effort for additional certificate European certificate gives added value, especially for international students! Diagnosis of existing quality assurance concept and inspection of existing procedures Promotion of the programmes (broader scope)

Your Accreditation: Pitfalls

Things tend to get hectic towards deadlines Especially for all dates involving multiple parties Coordination of multiple parties must not be underestimated Tracking of changes in documents and other data is indispensible Different pre-existing procedures need to be unified/harmonized e.g. existing course information system (central information depot) e.g. central administration -vs- local department procedures and best practices Showing different campuses on audit is hardly possible

Your Accreditation: Recommendations

Start early! Involve all parties with sufficient prep. Time Establish central coordination point (on two levels max!) one on the top level one for each party involved Propagate proper formulation of learning outcomes!

Background (1)

Università degli Studi di Milano •Founded in 1924, is the second biggest Italian university •Students ≈ 60.000 •Professors ≈ 2200 •9 schools

– Agricultural – Mathematics, Physics and Natural Sciences – Law – Letters and Philosophy – Medicine – Pharmacy – Political Sciences – Sciences and Veterinary Medicine

Background (2)

• The Dipartimento di Informatica e Comunicazione (DICO, Dept. of Informatics and Communication) and the Dipartimento di Scienze dell'Informazione (DSI, Dept. of Information Sciences), both located in Milan offer

– Bachelor Degrees: Informatics, Digital Communication,

Music Informatics – Master Degrees: Informatics, Informatics for

Communication

Degree submitted for Review (1) Bachelor Degree Informatica •Besides the Bologna three-year structure, which was introduced in the academic year 2002/2003, the study programme has been recently revised in order to conform it to the national regulation Ministerial Decree (“Decreto Ministeriale”) n.544 of 2007 (DM 544/07) concerning quality of Higher Education

•Offered starting from the academic year 2008/2009

Degrees submitted for Review (2)

Bachelor Degree Informatica

•Provides a sound basic knowledge of the major areas of Informatics along with a good grasp of their methodologies and technologies, with the goal of providing an adequate and modern preparation for the different applications in the field. The degree has two curricula, the preliminary curriculum, aimed at providing basic training leading to the master degree, and the professional curriculum, aimed at preparing attendees towards immediate employment. The two curricula, while clearly differentiated, share broad common ground that preserves the cultural coherence of graduates.

•Study mode – full time

•Programme duration – 3 years (6 semesters), 180 ECTS CP

•Enrolment ≈ 160 students per year; Staff 47

•Students pay tuition fee amount of 2000 EUR per academic year

Preparation / Workplan

• Date of Request for accreditation – May, 2010

• Milestones of preparation – Set up internal working group – October 2010

– Identification of data sources and collection of data – October, November 2010

– Writing of draft self-assessment report in English – November-December 2010

– Revision of self-assessment report – January, 2011

– Fill out of LO matrix – January 2011

– Presentation of self-assessment report at the Council of Studies – February 2011

• Date Submission of draft self-assessment report – December 21, 2010

• Date Submission of final Version – January 27, 2011

• Date of auditing visit – March 29-30, 2011

• Date of decision – June 27, 2011

Self-Assessment Report (1) • Stakeholders involved in preparing the Self-Assessment

Report – the teaching staff, the programme coordinators, the accreditation

responsibles, some students, some administrative personnel, the teaching manager, the central offices of our university for aspetcts related to graduates placement, teaching assessment, diploma supplement, regulations and so on.

• Accreditation responsibles were the main actors involved in the production of the self-assessment report, by collecting, revising, and merging all the information coming from all involved stakeholders.

• Extent / appr. nr. of pages Report – 35 • Extent / appr. nr. of pages Annexes – 150 (12 annexes)

Self-Assessment Report (2) • 3 biggest challenges in report writing

– Understanding the meaning of “learning outcome” in the EQANIE accreditation framework and identification of LO in our programme designed according to a different framework (i.e., Dublin framework).

– Identification of the mappings between the three abstraction levels (overarching objectives (strategy), EURO-INF learning outcomes and modules learning outcomes) required in order to fill-in all involved tables, whose size is not easily manageable.

– Translation of involved institutional regulations/resolutions documentation of UNIMI

Benefits of the accreditation experience

• The accreditation process has been considered by the entire UNIMI staff as an opportunity for improving the quality of the Bachelor Degree Informatica.

• The experience turned out to be very useful for defining with more precision the main objectives of the programme as well as for individuating improvements and weaknesses.

• The discussions and exchange of ideas we had with the colleagues from the audit team were very productive, as we had the opportunity to hear about similar experiences in Europe, and thus a yardstick for better evaluating our programme.

Your Accreditation: Pitfalls • Read carefully the documentation for a correct understanding of the

notion of learning outcome which is key for the whole process. The entire documentation the term learning outcome is used for referring both to the EURO-INF learning outcomes and to module learning outcomes, and not always the context helps in individuating the correct semantics.

• Compare the criteria used to design the study programme with the EQANIE criteria based on the notion of learning outcome. A non negligible effort is required to rethink the study programme objectives in term of EQANIE learning outcomes if the study programme has been conceived referring to a different standard.

• The accreditation process requires a joint effort of all involved stakeholders. Getting people involved in the process and keeping them motivated is not easy, also because the accreditation activities come on top of the ordinary activities in charge to the teaching staff.

Your Accreditation: Recommendations • To remark the “officiality” of the audit visit and the

importance of the accreditation procedure to involved people since the very beginning and periodically, to increase people motivation and commitment.

• To deliver template documentation including examples of descriptions of the key accreditation information to be provided (e.g., module handbook, learning outcomes) in order to get homogeneous documentation from the staff.

• To focus very soon on understanding the mappings between the three abstraction levels (overarching objectives (strategy), EURO-INF learning outcomes and modules learning outcomes) in order to produce since the beginning effective documentation.

Riga Technical University Background

• Riga Technical University (previously named Riga Polytechnical Institute) was founded in 1862

• 8 faculties – Faculty of Architecture and Urban Planning – Faculty of Civil Engineering – Faculty of Computer Science and Information Technology – Faculty of Electronics and Telecommunications – Faculty of Engineering Economics and Management – Faculty of Materials Science and Applied Chemistry – Faculty of Power and Electrical Engineering – Faculty of Transport and Mechanical Engineering

• Number of students – 15735 • Staff – 764 • Professors – 135

Faculty of Computer Science and Information Technology (1)

• Founded in 1961

• Number of students ≈ 1200

• Number of graduates – 3000+ engineers till 1991

– 2000+ bachelors

– 800+ masters

– 1000+ engineers

– 50+ doctors of engineering

• Staff – 93 (32 professors and associated professors)

Faculty of Computer Science and Information Technology (2)

• Study programmes – Automation and Computer Engineering offered by Institute of

Computer Control, Automation and Computer Engineering

– Information Technology offered by Institute of Information Technology

– Computer Systems offered by Institute of Applied Computer Systems

– Intelligent Robotic Systems offered by Department of Systems Theory and Design of Institute of Applied Computer Systems

– Business Informatics offered by Institute of Applied Computer Systems and Institute of Information Technology

– Financial Engineering offered by Department of Probability Theory and Mathematical Statistics


Bachelor’s Programme “Computer Systems” • Provides a basic education in computer science and adequate preparation

for further studies in the consecutive Master degree programme • Study mode – full time • Programme duration – 3 years (6 semesters), 181.5 ECTS CP • Programme structure:

– Mandatory modules: • General subjects (Mathematics, Physics, Electronics and Electrical Engineering, Discrete Mathematics,

etc.)

• Basic subjects of computer science (Programming Languages, Operating Systems, Introduction to Computer Architecture, Applied Software, Data Structures, Object-Oriented Programming, Database Management Systems, Computer Networks, Foundations of Artificial Intelligence, etc.)

• Compulsory subjects (Methods of Systems Theory, Computer Organization and Assembly Language, Technology of Large Databases, System Analysis and Knowledge Acquisition, Software Engineering, etc.)

– Restricted electives (correspond to narrow concentration areas) – Free electives

• Enrolment – 120-130 students each year • Around 93% of the students do not pay tuition fees (receive a full state-

financed scholarship) • Awarded degree: BA Eng.Sci. (computer control and computer science)


Master’s Programme “Computer Systems”

• Provides deep knowledge for problem solving in the area of computer science with focus either on software engineering/programming, computer system development or systems analysis and prepares graduates for continuing their studies in doctoral degree programmes

• Study mode – full time

• Programme duration – 2 years (4 semesters), 121.5 ECTS CP

• Programme structure similar with the Bachelor’s Programme

• Enrolment – 50-60 students each year

• 100% of the students do not pay tuition fees (receive a full state-financed scholarship)

• Awarded degree: MA Sc.Eng. (Computer Systems)

Preparation / Workplan

• Date of Request for accreditation – April 15, 2010 (October 25, 2010 when submitted programmes were changed from professional to academic)

• Milestones of preparation – Set up internal working group – October 2010

– Identification of data sources and collection of data – October, November 2010

– Writing of draft self-assessment report in Latvian – November 2010

– Writing of draft self-assessment report in English – December 2010

– Addition of self-assessment report – January, February 2011

– Fill out of LO matrix – February 2011

– Translation of regulating documents from Latvian to English – February, March 2011

– Presentation of self-assessment report at the Council of Institute of Applied Computer Systems – March 2011

• Date Submission of draft self-assessment report – December 22, 2010

• Date Submission of final Version – March 7, 2011

• Date of auditing visit – April 5-7, 2011

• Date of decision – July 12, 2011


• Stakeholders involved in preparing the Self-Assessment Report – Taking into account that the self-assessment report was written on the

basis of annual self-assessment reports the internal working group is the same – one person appointed by the programme director for data collection and one person responsible for written version of the self-assessment report. Procedure is managed by heads of involved departments and supervised by director of the study programme

• Extent / appr. nr. of pages Report – 35

• Extent / appr. nr. of pages Annexes – 518 (including Module handbook – 256 p., Teaching capacity – 39 p. generated from RTU information systems, and External Regulations – 68 p. which are not translated in English)


• 3 biggest challenges in report writing? – Some lack of understanding how to fill out required Table 3

and Table 4 as well as Euro-Inf Learning Outcomes

– Translation of normative documents of Riga Technical University among which despite the existence of Foreign Students Department even Terms of Enrolment and Regulations of Studies are not fully translated in English

– It is impossible to identify individual versions of document “Procedural principles for the accreditation” contents of which are different

Benefits of the accreditation experience (1)

• Regardless of fact that we already have experience in writing self-assessment report (the study programme Computer Systems has QUESTE (Quality System of European Scientific and Technical Education) certificate), the EQANIE requirements give us new points of view about the programme and auditing visit provides neutral external evaluation of programme. Positive decision about quality of the study programme Computer Systems strengthens our belief that the programme proceeds in right direction and in general corresponds to similar study programmes in informatics in Europe

Benefits of the accreditation experience (2)

• The necessity to write the self-assessment report pushed us to revise goals of the study programme. As a result we discovered lack of conformity between goals of the study programme and several courses.

• We found several courses which descriptions are rather poor, in particular, formulation of goals and learning outcomes

• During preparation of self-assessment report we found several new information systems of RTU which allow generation of needed data but are not known for staff and students

• Academic staff started to show more interest about quality issues

• Accreditation stimulates estimating of staff workload at the level of whole programme

Your Accreditation: Pitfalls

• Collecting all necessary data from the staff (especially from other departments and faculties) always is a painful process

• Preparations for accreditation, the main part of which is writing of self-assessment report of good quality, may discover different problems within institution. Trying to start discussions probably may not lead to consensus during accreditation

• Module handbook in result may be inconsistent due to the fact that different modules are written by different people (especially it is the case when these people are coming from different departments). To get a consistent module handbook practically turns out to be a separate project

Your Accreditation: Recommendations (1)

• Start with exploration of available information. In many cases information can be used as it is available, i.e., only with little adjustments

• Examine the existing documentation at your university to know what is already done by the central administration and/or centralized information system maintainers

• Don’t hesitate to ask for help the EQANIE Secretariat at an early stages of the preparation stage – it will avoid misunderstanding with documents and will save a lot of time

Your Accreditation: Recommendations (2)

• Participate in at least one seminar organized by EQANIE Secretariat, in particular, on Learning Outcomes (LO) being offered in the future

• Start curriculum analysis with LO matrix which can be completed using the module handbook. This allows you to evaluate the quality of module descriptions

• Accreditation is not “one person’s show“, but at the same time the staff feels more committed if there is “an accreditation face”, i.e., someone who is a permanent person with whom to speak in the team

• Quality assurance is an ongoing process. The same team would have to continue work after the accreditation

Documents

Accreditation from the perspective of three European ... Conference/Castano... · Nov 2010 review prelim. decision and prepare response . Preparation / Workplan (1) ... Faculty for