Best Practices in Washington Accord Signatories With reference to the accreditation criteria, systems and procedures

Liew, C. P., Puteh, M., Mohammad, S. Universiti Teknologi Malaysia

Malaysia

This paper discusses the practices in selected Washington Accord Signatories in terms of accreditation criteria, systems and procedures with reference to Malaysia. It aims to draw a conclusion on the practices that could improve the operations of accreditation and increase the objectivity of awarding accreditation. This comparative study was presented to the Engineering Accreditation Council Department, Malaysia in August 2013.

Keywords—accreditation; practices; consistency; competency; decision; objectivity

I. INTRODUCTION The Washington Accord has grown from six signatories in

1989 to a well-sought-after organization with Russia being the recent 15th signatory in 2012. The Board of Engineers, Malaysia secured the full signatory of the Washington Accord for Malaysia in June 2009 and became the 13th signatory. Malaysia’s entry to the Accord marked a significant milestone of the quality of its engineering education whereby its engineering graduates are recognized as meeting the academic standards for engineering practice in other signatory countries. Engineering degree programmes are accredited by the Engineering Accreditation Council (EAC), a body delegated by the Board of Engineers, Malaysia.

With the growing the number of signatories, it is worthwhile to study the practices adopted by them in terms of accreditation criteria, systems and procedures.

This comparative study aims to identify noteworthy practices adopted by selected Washington Accord signatories with reference to Malaysia where applicable; and to draw a conclusion on the practices that could improve the operations of accreditation and increase the objectivity of awarding accreditation.

II. DEFINITIONS The definitions used by signatory countries are differ from

one to another in classifying the shortcomings observed in a programme under accreditation evaluation.

Table 1 [3] [4] shows the definitions used by the Accreditation Board for Engineering and Technology Inc. (ABET), the United States of America and the Engineering Accreditation Council, Malaysia (EAC Malaysia).

TABLE 1.

Definition of Shortcomings and Observation by ABET

Definition of Shortcomings and Opportunity for Improvement by EAC Malaysia

Shortcomings Shortcomings

Def

icie

ncy

A Deficiency indicates that a criterion, policy, or procedure is not satisfied. Therefore, the program is not in compliance with the criterion, policy, or procedure.

Wea

knes

s A Weakness indicates that the programme/institution has failed the criteria set under the accreditation manual

Wea

knes

s

A Weakness indicates that a program lacks the strength of compliance with a criterion, policy, or procedure to ensure that the quality of the program will not be compromised. Therefore, remedial action is required to strengthen compliance with the criterion, policy, or procedure prior to the next review.

Con

cern

A Concern indicates that a program currently satisfies a criterion, policy, or procedure; however, the potential exists for the situation to change such that the criterion, policy, or procedure may not be satisfied.

Con

cern

A Concern indicates that the programme/institution has not failed the criteria set under the accreditation manual, however, if left unchecked it may lead to failure at a later date.

Observation Opportunity for Improvement An Observation is a comment or suggestion that does not relate directly to the current accreditation action but is offered to assist the institution in its continuing efforts to improve its programs.

An Opportunity for Improvement refers to further improvement that an institution could consider despite having already satisfied the minimum requirements of the accreditation manual.

III. 1989 - THE ORIGINAL SIX In this section, the major practices of the Accreditation

Board for Engineering and Technology Inc., the United States of America and the Canadian Engineering Accreditation Board, Canada will be discussed. The United States of America and Canada are part of the original six signatory countries which established the Washington Accord in 1989.

A. Accreditation Board for Engineering and Technology, Inc.(ABET) The accreditation criteria, policy and procedure (i.e., the

model) implemented by ABET is arguably one of the most

2014 International Conference on Teaching and Learning in Computing and Engineering

978-1-4799-3592-5/14 $31.00 © 2014 IEEE

DOI 10.1109/LaTiCE.2014.60

278

popular models benchmarked by the Washington Accord signatory countries. ABET was founded in 1932, and has accredited 3,278 programmes at 671 institutions in the United States and 324 programmes at 64 institutions in 23 other nations as of Sep. 2012 [1]. These numbers clearly show that institutions in the United States and other nations have a high regard for ABET accreditation.

ABET has four commissions that lead and conduct its accreditation activities. The commission relevant to the context of this study is the Engineering Accreditation Commission, responsible for the accreditation of engineering programmes.

The authors identified the following practices by EAC-ABET which may be comparatively different to the Engineering Accreditation Council, Malaysia (EAC Malaysia):

• Accreditation Visits and Decision Meeting – The Engineering Accreditation Commission of ABET carries out its accreditation visits within a stipulated period in the current year, i.e., September to December and an annual accreditation decision meeting is carried out to determine on the type of actions resulting from these visits in the following year (i.e., month of July) [2]. In contrast, EAC Malaysia carries out accreditation visits almost throughout the year and conducts its accreditation decision meetings thrice a year. While both approaches have their merits and demerits, the former allows a better planning of resources in terms of secretariat, training of evaluators, implementing improvements, and so on, and the latter provides speedier accreditation decisions to the institutions.

• Types of Accreditation Actions – The types of accreditation actions adopted by ABET and EAC Malaysia are illustrated in Table 2 [3] and Table 3 [4] respectively. The accreditation actions adopted by ABET are of the duration of six, four, two and zero year with a complete cycle of six years. In general, programmes under general review (i.e., new cycle accreditation in the Malaysian context) are awarded six or two years accreditation; six years accreditation is awarded to programmes without any reported deficiencies or weaknesses; and a common two-year accreditation is awarded for programmes with either deficiencies or weaknesses. The key point for this common two-year accreditation is the terminologies used, i.e., “Interim Visit/Report” for programmes with weaknesses and “Show Cause” for programmes with deficiencies. The authors believe that these terminologies are more reflective in terms of urgency of remedial actions expected from the institutions while providing sufficient time for implementation.

In this regard, EAC Malaysia accords one year accreditation to demonstrate the urgency for some shortcomings to be addressed immediately which may place the institutions in greater pressure due to the shorter duration.

TABLE 2.

Types of Accreditation Actions by ABET

Type Definition Next Action Duration (Years)

Next General Review

No Deficiencies or Weaknesses None 6

Interim Report

No Deficiencies but one or more Weaknesses

Progress Report 2

Interim Visit No Deficiencies but one or more Weaknesses

On-site Review 2

Show Cause Report One or more Deficiencies Progress

Report 2

Show Cause Visit One or more Deficiencies On-site

Review 2

Report Extended Successful Interim Report Not applicable 2 or 4

Visit Extended Successful Interim Visit Not applicable 2 or 4

Show Cause Extended

Successful Show Cause Report/Visit Not applicable 2 or 4

Not to Accredit

Deficiencies still present after Show Cause Appeal 0

TABLE 3.

Types of Accreditation Actions by EAC Malaysia

Type Definition Next Action

Five years accreditation No Weaknesses or Concerns None

Five years accreditation with interim report*

Satisfied the minimum requirement with a few minor and isolated concerns

Interim Report

Three years accreditation with two years extension*

Satisfied the minimum requirement with a few minor and isolated concerns

Interim Visit

Two years accreditation

Satisfied the minimum requirement with a list of major and related concerns

Interim Visit

One year accreditation

Satisfied the minimum requirement with numerous Concerns and previous Concerns not addressed adequately

Interim Visit

Decline accreditation or zero year

Non-compliance with the minimum requirement

Reappli-cation

* Three years accreditation with two years extension is generally preferred over five years accreditation with interim report as a few concerns are difficult to equate and difficulty to

withdraw accreditation upon unsatisfactory submission of interim report

• Consistency of Accreditation Actions – There is a presence of consistency committee under the Engineering Accreditation Commission of ABET which is responsible for reviewing the accreditation actions determined by the commission. In addition, a member of the executive committee is assigned to reviews and edits the draft and final statements to ensure that the content is consistent with ABET’s policies. Consistency of accreditation actions is also supported by ABET’s well-structured calendar that allows training programmes to be conducted for new evaluators and team chairs, and prepares online training courses such as refresher programme to expose evaluators to the latest updates on criteria, processes, and procedures. The learned societies are responsible for the training for

279

programme-specific criteria for their respective evaluators.

In this regard, EAC Malaysia organizes meetings with team chairs to harmonize recommended accreditation actions before presenting them to the council and respective deans of institutions are also invited to its accreditation decision meetings. Training programmes for new evaluators are often conducted by a learned society in a periodic manner.

• Centralised and Well-structured Events – ABET organizes: yearly symposium for faculty and administrators to promote peer-to-peer and expert interaction around programme assessment; programme assessment workshops to develop programme assessment skills; programmes for faculty and administrators to develop new knowledge and skills to be an effective assessment leader through its Institute for the Development of Excellence in Assessment Leadership; and webinars that cover a wide range of topics from planning on-site visits to developing rubrics for assessment.

• Assessment and Evaluation of Program Educational Objectives (PEOs) – ABET has removed the stringent requirement on programmes to provide assessment and evaluation results on PEOs; and institutions are required to explain the review process that aligns the PEOs with their mission statements, stakeholders’ needs and applicable criteria.

B. Canadian Engineering Accreditation Board (CEAB) CEAB was established in 1965, and has accredited 271

programmes at 43 institutions as of Sep. 2012 [5]. It introduced outcomes-based assessment as part of its accreditation criteria in 2008 and allowed a transition period from 2009 to 2014. Effectively from June 2015, accreditation decision will be based on outcomes assessment on graduate attributes [6].

The authors identified the following practices by CEAB which be comparatively different to EAC Malaysia:

• The Engineering Graduate Attribute Development (EGAD) Project – The EGAD project is sponsored by the National Council of Deans of Engineering and Applied Science and Engineers, Canada; it is formed to assist the Canadian faculties and schools of engineering to implement the outcomes-based assessment during the transition period stipulated by CEAB.

• Accreditation Visits and Decision Meeting – Similarly to ABET, CEAB carries out its accreditation visits within a stipulated period in the current year, i.e., September to December and an annual accreditation decision meeting is carried out to determine on the type of actions resulting from these visits in the following year (i.e., month of June).

• Editor – Similar to ABET, a member of the accreditation board executive committee is assigned to review the reports from team chairs to ensure that the

content is consistent with the accreditation board’s policies [7].

• Assessment and Evaluation of Program Educational Objectives (PEOs) – This is not present in the accreditation criteria and procedures.

IV. 1990S Hong Kong, China and South Africa entered the

Washington Accord in 1995 and 1999 respectively. The major practices of the Engineering Council of South Africa (ECSA) will be discussed in this section.

ECSA was founded in 1932, and has accredited 99 programmes at 18 institutions through its Engineering Programme Accreditation Committee as of June 2013 [8].

The authors identified the following practices by ECSA which be comparatively different to EAC Malaysia:

• Types of Accreditation Actions – With reference to Table 4 [9], the accreditation actions adopted by ECSA are very much based on the reported deficiencies by the accreditation team: a full 5-year accreditation is accorded for programmes without deficiencies and; no more than 3 years accreditation for programmes with deficiencies. The latter will be verified through the modes of interim visit or interim report.

• Deficiencies and Concerns – ECSA expects that reported deficiencies and concerns must be addressed adequately; and it is part of the accreditation criteria [10].

• Assessment and Evaluation of Program Educational Objectives (PEOs) – Similar to CEAB, this is not present in the accreditation criteria and procedures.

TABLE 4.

Definition of Shortcomings by ECSA

Def

icie

ncy

A condition or combination of factors related to a programme that are not in conformance with accreditation criteria that prevent full-term accreditation being granted. A deficiency could result from the cumulative effect of a number of issues, each of which taken in isolation would not preclude accreditation. One or more deficiency precludes accreditation until the next regular visit and requires an interim evaluation of the programme.

Con

cern

A matter which an accreditation team believes adversely affects the quality of the programme but which does not preclude granting of accreditation. Concerns must be satisfactorily resolved by the next regular or interim visit. A concern not resolved by the next visit may then be judged to be a deficiency.

Com

men

t

Communicates to the academic unit impressions of the team, commendations or constructive criticism on negative factors which are not classified as deficiencies or concerns.

280

V. 2000S - ASIA PACIFIC The year 2005 to 2009 witnessed the interest of Asian

countries getting entries into the Washington Accord, namely, Japan, Singapore, Korea, Chinese Taipei and Malaysia. These Asian countries demonstrate their resolutions to practice the equivalent accreditation as their Western counterparts.

The major practices of the Accreditation Board for Engineering Education of Korea, Japan Accreditation Board for Engineering Education and the Institute of Engineering Education Taiwan and will be discussed in this section.

A. Accreditation Board for Engineering Education of Korea (ABEEK) ABEEK was founded in 1998, and has accredited 326

programmes at 54 institutions in 2012 cycle.

The types of accreditation actions are similarly to ABET; and the assessment and evaluation of PEOs are present as part of accreditation criteria.

The notable practice by ABEEK is the four-level of consistency check to reduce the subjectivity in awarding accreditation. These levels are [12]:

1. A chairman for each accreditation evaluation panel who is responsible of reviewing the findings of the panel members.

2. Evaluation Report Consistency Committee which is responsible for reviewing the evaluation outcomes given by each panel of evaluators.

3. Consistency Committee on Program Criteria which is responsible for reviewing and adjusting the evaluation outcomes to ensure consistency among the programmes within the same discipline.

4. Consistency Committee on Institutions and Accreditation Cycles which is responsible for reviewing and ensuring consistencies between all institutions of higher education being evaluated within the given year and across different years.

B. Japan Accreditation Board for Engineering Education (JABEE) JABEE was established in 1999, has accredited 466

programmes at 169 institutions as of 2012. There are potentially 1,500 engineering programmes at about 250 institutions that could be accredited.

In general, similar to ABET, JABEE conducts yearly workshops to programme evaluators, before its regular visit period. However, the assessment and evaluation of PEOs is not part of the accreditation criteria.

Similar to ABEEK, JABEE practices similar structure for consistency check with the absence of Level 2 and Level 3 carried out by engineering learned societies [12].

C. Institute of Engineering Education Taiwan (IEET) IEET was founded in 2003, has accredited 162 programmes

at 56 institutions in 2012/2013 cycle.

IEET conducts yearly briefing or workshops to institutional representatives in addition to programme evaluators before its regular visit period; and the assessment and evaluation of PEOs is part of the accreditation criteria.

VI. CHALLENGES IN MALAYSIA Malaysia is a relatively new to the Washington Accord

having secured the full signatory of the accord in 2009 and need to deal with a set of challenges towards excellence.

Table 5 is constructed to illustrate some of these challenges. It shows the frequency of Electrical and Electronic engineering degree programmes awarded accreditation classified by number of years between 2009 and 2012 at thirty (30) higher education institutions [12].

Two-year accreditation award is rather a common decision among programme evaluators. It is interesting to note that between 2009 and 2012, 69.0 percent of the institutions were only granted a 2-year accreditation whereas only 19.6 percent of Malaysian Electrical and Electronic Engineering programmes were accorded with 3-year accreditation; and full accreditation (5 years) was not awarded during the three year period.

TABLE 5.

The frequency of engineering degree programmes (*Electrical and Electronic engineering) awarded accreditation classified by the number of years between 2009 and 2012 at thirty Malaysian higher education institutions

Number of Years Awarded Occurrence Occurrence (%)

5 0 0.0

4 Not relevant Not relevant

3 33 19.6

2 116 69.0

1 19 11.3

0 or Decline Accreditation Information not available

Information not available

*Electrical and Electronic engineering includes related disciplines such as electrical, electronics, communication, control and instrumentation, mechatronics, computer and biomedical

The common decision of 2-year accreditation and no programme was accorded full accreditation are likely due to the underlying issues with regards to the accreditation assessment in engineering programmes. These issues are not limited to the following [12]:

• Accreditation assessment is fault-finding in nature which resulted in extreme low number of programmes being accorded full accreditation.

• The competency level of programme evaluators need to be raised due to conflicting advices and recommendations among them.

• The requirements of accrediting engineering programmes are not clearly defined for and understood by institutions.

281

The abovementioned issues highlight the importance of: training of programme evaluators and awareness of accreditation requirements among institutions.

VII. RECOMMENDATIONS FOR NEW ENTRANTS It is noted that all accreditation bodies in this study

recognized the importance of a well-structured and systematic training programme to increase the level of competency of programme evaluators and team chairs. It is recognised as the first check point towards the objectivity and consistency of awarding accreditation.

New entrants to the accord implement multiple levels of consistency check on the accreditation actions recommended by programme evaluators, likely due to the lower level experience compared to well-established accreditation bodies. They are also noted for their continuing awareness campaign of accreditation requirements to institutions.

Well-established accreditation bodies such as ABET and CEAB tend to create resources to assist institutions to produce graduates that meet the academic standards for engineering practice. The EGAD project, a notable initiative, sponsored by the National Council of Deans of Engineering and Applied Science and Engineers, Canada to assist the Canadian faculties and schools of engineering to implement outcomes-based assessment.

Accreditation actions that are reflecting the urgency of the actions expected from an institution, for example, “Show Cause” while providing sufficient time for remedial actions appear to be effective in driving changes.

ACKNOWLEDGMENT This is an ongoing research project funded by the Ministry

of Education Malaysia under the Exploratory Research Grant Scheme (ERGS). Vot no. 4L035.

REFERENCES [1] 2012 ABET Annual Report, ABET, Baltimore, MD, Sept. 2012. [2] D. Briedis and D. Warder, “Preparing engineering self-study of

accreditation & accreditation Myths,” in 2013 ABET Symp., Portland, Ore., 2013.

[3] Accreditation Policy and Procedure Manual, ABET, Baltimore, MD, Oct. 2013.

[4] Determining Accreditation Decision, 2nd ed., Engineering Accreditation Council, Malaysia, Kuala Lumpur, MY, 2012.

[5] Engineers Canada. (2012, Dec. 6). Engineering Education Accreditation in Canada [Online]. Available: http://www.engineerscanada.ca/.

[6] Accreditation Criteria and Procedures, Canadian Engineering Accreditation Board, Ottawa, ON, 2012.

[7] Manual of Accreditation Procedures, Canadian Engineering Accreditation Board, Ottawa, ON, Mar. 2009.

[8] University Degrees Accredited as meeting the Educational Requirement for Registration as a Professional Engineer, Rev. 16, Engineering Council of South Africa, Johannesburg, ZA, July 2013.

[9] Policy on Accreditation of Engineering Programmes Meeting Stage 1 Requirements, Rev. 1, Engineering Council of South Africa, Johannesburg, ZA, Feb. 2008.

[10] Criteria for Accreditation of Engineering Programmes Meeting Stage 1 Requirements, Rev. 1, Engineering Council of South Africa, Johannesburg, ZA, Feb. 2008.

[11] Criteria for Accrediting Engineering Programs, Accreditation Board for Engineering Education of Korea, `Seoul, KR, Unknown.

[12] C. P. Liew et al, “A New Assessment Strategy for Accrediting Engineering Programmes for Malaysia,” in 2013 Research and Engineering Education Symp., Kuala Lumpur, MY, 2013.

282