Performance Measure of Academic Departments Using Data Envelopment Analysis

Zuraida Alwadood, Norlenda Mohd Noor, Mohd Fadzil Kamarudin Faculty of Computer and Mathematical Sciences

Universiti Teknologi MARA Selangor, Malaysia

Abstract— Performance efficiency of educational institution should be continuously monitored and improved to ensure that qualified manpower and research are produced efficiently in meeting the university standard of intellectuals. With a systematic performance monitoring system, resources will be channeled sufficiently and tasks may be completed effectively. Therefore, the main objective of the research is to use data envelopment analysis (DEA) to measure the performance efficiency of the academic departments in a faculty in a public university in Malaysia. Two inputs and three outputs which strongly influence the efficiency of the academic departments were selected. The second objective is to rank the academic departments according to their performance efficiencies. The final objective is to uplift the efficiency score of the less efficient department. In general, all academic departments in the faculty recorded a high level of efficiency of above 90%. The academic departments with perfect efficiency score were considered as the benchmark departments. By means of the reference set obtained in the output, the improvement strategies recommended to the less efficient academic departments are to reduce its Input 1 and to increase its Output 3.

Keywords- performance efficiency, data envelopment analysis, academic department, input and output, reference set.

I. INTRODUCTION Performance efficiency played an important role in

assessing an organization ability to achieve targets. The efficiency measurement is applicable to all types of business including education industry. As the number of educational institution increases dramatically, performance efficiency is currently used as one of the criteria to ensure that the expected level efficiency is met. Academic departments will have to focus on teaching, research or scholarship so as to achieve the university’s mission [1]. There are major programs of study are located, courses offered, and research conducted in any particular academic department. This research focused on one of the academic faculties in a local university which has six academic departments and currently offers tertiary education programs.

The performance of an academic department as well as the quality of outputs contributes to achieve a certain level of efficiency. Without monitoring the performance efficiency systematically, the faculty will not be able to estimate the optimum amount of resource that should be channeled to its

academic departments. Beside this, it would not know if the facilities offered are sufficient enough or need immediate improvements, so as to ensure that tasks are completed effectively. In a wider view, performance efficiency level would generally be the indicator in comparing the ranking of academic departments globally.

This study is aimed to investigate the efficiency of six academic departments in the selected faculty, in an effort to identify the best practices and benchmark. The efficient academic department will be used as a benchmark or reference, in the attempt to improve the performance efficiency of the less efficient ones. The main objective is to measure the relative efficiency of academic departments. Then, the studies centers will be ranked based on their efficiency level. Finally, some improvement strategies will be suggested to the less efficient department to uplift their efficiency to a satisfactory level. In this research, the selection of the outputs is made based on the input criteria chosen.

A systematic performance measure is crucial for an organization in the attempt to foster an environment of continuous improvements and ensure successful implementation of strategic plans. There are many literatures focusing on the strategies used in developing performance measures for educational institutions. A study done by [2] used the integrated strategic approach for evaluating engineering schools. An integrated framework for self-assessment was emphasized in [3] to assess the performance measure at the College of Engineering and Petroleum in Kuwait University. There are variety of performance measure studies in the literature, such as stochastic frontier techniques, Bootstrap method, cray performance analysis and super-efficiency analysis, to name a few.

In spite of the bulk methods done on performance measure, this paper has chosen data envelopment analysis (DEA) technique to be used in examining the efficiency of the academic departments. This is due to the fact that DEA is one of the most suitable methods as it involved transforming multiple inputs into multiple outputs [4]. It is a mathematical programming technique that helps to identify the reference sets for inefficient decision making units (DMU). Besides being able to suggest efficiency improvements, it also helps in utilizing and allocating scarce resources. Due to these facts, DEA has becoming a preferable choice of technique for measuring the efficiency of education institutions.

2011 IEEE Symposium on Business, Engineering and Industrial Applications (ISBEIA), Langkawi, Malaysia

978-1-4577-1549-5/11/$26.00 ©2011 IEEE 395

∑

∑

=

== m

jjj

s

kk

xu

yvA

10

1k0

0 Maximum

1

subject to

10

1k0

≤∑

∑

=

=m

jjj

s

kk

xu

yv

(2)

where,

ju kv

ij xiky

npnimjsk

j

k

ji

ki

input given toweight output given toweight

DMUby producesinput ofamount DMUby produces output ofamount

,,2,1,,2,1,,2,1,,2,1

==

==

====

∑=

=s

kpk yvA1k

0 Maximum

∑ ∑

∑

= =

==

≤−s

k

m

jjpjkpk

m

jjpj

xuyv

xu

1 1

11

0

subject to

(3)

where,

0,,,2,1,,2,1,,2,1

≤===

jk xvwpmjsk

DEA has become an important tool for evaluating and improving the performance of manufacturing and service operations. A number of researches were conducted in measuring performance efficiency such as performance evaluation and benchmarking of schools, hospitals, bank branches, production plants, etc [5]. Reference [6] used DEA method to evaluate engineering and technical educational institution. In [7], DEA was used to examine the relative efficiency in the production of research of Chinese regular universities in 2003 and 2004. The efficiency score recorded is above 90% and this is higher in comprehensive universities compared to specialist universities.

This technique is also applied to a data set of higher education institution in England using data for the year 2000/01 [8]. Technical and scale efficiency score in the English higher education sector appear to be high on average. In [9], DEA models that utilize hierarchical structures of input–output data to enable them in handling large numbers of inputs and outputs were developed. They presented two approaches in a pilot evaluation of 15 institutes for basic research in the Chinese Academy of Sciences using the DEA models.

An effective and successful academic department management will be able to increase the qualities of output if the utilization of the available resources is maximized [10]. Therefore, the measure of the performance of the academic departments has to be implemented in order to maintain their reputation and qualification in producing the outputs. Hence, one needs to focus on the importance of considering the university’s mission and vision while selecting the outputs and inputs, so that the output produced will be directed towards achieving the university intended measures.

II. METHODOLOGY Factors that influence the efficiency of each department

are identified to be the inputs or the outputs [11]. In a broader sense, these factors must be relevant and directly aligned with the goals and the objectives of the department [1]. This research considers the objectives of the university in order to identify and classify the factors that influence the performance efficiency of the academic departments.

In DEA method, the measurement of efficiency is simply the ratio between the inputs and outputs. Model equation (1) is proposed by [5] to measure the efficiency of DMUs. The efficiency score in the presence of multiple input and output factors is defined as

inputs of sum Weightedoutputs of sum WeightedEfficiency =

(1)

Assuming that there are n DMUs, each with m inputs and s

outputs, the relative efficiency score of a test DMU is obtained by solving the model equation (2).

Reference [5] stated that, the fractional program shown by

the model equation (2) can be converted to a linear programming model (3).

Five input measures were highlighted in [1], namely

faculty utilization, course offering, quality of incoming students, quality of graduate students and support staff capabilities in their study. In selecting the most appropriate

396

∑=

=3

1k0 Maximum kpk yvA

∑ ∑

∑

= =

==

≤−3

1

2

1

2

11

0

subject to

k jjijkik

jjij

xuyv

xu

(4)

where,

ju kv

ij xiky

iwp

j

k

ji

ki

input given toweight output given toweight

DMUby producesinput ofamount DMUby produces output ofamount

6,,2,1,,2,1

==

==

==

∑

∑

=

== m

jjj

kk

Di

xu

yvA

10

3

1k0

Maximum

(5)

input for measuring performance, an association matrix was used to identify the association strength of each input towards the university objectives. Upon performing a similar analysis, this study has selected two inputs that strongly influence the objectives of the faculty. The inputs are identified as department utilization and program offering.

The indices used to measure department utilization are the department to student ratio, average student credit hour per department and number of graduate students. Program offering refers to the number of courses offered by each academic department.

Based on the same literature, three outputs were identified as the most significant outputs for any academic department. The outputs selected are defined as the quality of graduate, quality of research and quality of staff service. The quality of graduates is reflected by the percentage of employers surveyed who satisfied with the task completed by the graduates, percentage of alumni surveyed who satisfied with their current position and the Cumulative Grade Point Average (CGPA) for graduating students. The indices supporting the quality of research are the number of journal publication per department, number of conference papers per department, number of research grant per department and the number of graduate students supervised per department. The quality of staff service is measured by the number of short courses organized by departments, the percentage of staff participating in short courses, the number of consultancy jobs per department and the percentage of staff engaging in industrial consultancy. All data on the indices mentioned are based on the year 2010.

Once the number of inputs and outputs are identified, the model equation (3) can now be rewritten as equation (4).

Upon running the model on QM For Windows, the weights of each output and input were generated. Those values were then be substituted into equation (5).

Finally, the efficiency of the department ADi was

successfully obtained. The similar process was repeated for the other five academic departments.

III. RESULT AND DISCUSSION In general, all academic departments in this faculty

recorded a high level of efficiency. Table I shows the rank of the academic department according to its performance efficiencies. Two departments, D5 and D6 have performance efficiency of 100 percent. They are followed by department D1 (98.75 percent), D3 (96.63 percent), D4 (96.41 percent) and finally D2 with 91.96 percent of performance efficiency.

TABLE I. THE RANKING OF PERFORMANCE EFFICIENCY OF ALL ACADEMIC DEPARTMENTS UNDER STUDY

Department Relative Efficiency D5 D6 D1 D3 D4 D2

100% 100%

98.75% 96.63% 96.41% 91.96%

The concept of DEA stated that the maximum efficiency of

the DMU can be obtained if the resource allocations are wisely utilized, hence increase the production of the outputs. Table 1 indicates that the inputs of D5 and D6 have been efficiently utilized and the outputs produced are reasonably proportionate to the inputs available. Therefore, D5 and D6 are the most efficient departments compared to the other departments in the faculty. Due to this reason, they will be considered as the benchmarks.

By looking at the other end, D2 is the least efficient department because it only achieved 91.96 percent of efficiency level. The department has large inputs, i.e. department utilizations and program offering. Unfortunately, the performance efficiency is not satisfactory. The outputs produced are not proportionate to the amount of inputs available. It lacks of quality of graduates and the quality of research besides the quality of staff service does not meet the expectation.

The targeted efficient level can be obtained if the department knows the optimum amount of input and output that should be used and produced, respectively. Upon giving the efficiency level of each academic department, DEA also provides a reference set and its respective weight of dual values. References set consist of efficient DMUs and serve as basis in improving the efficiency of the less efficient DMU. In the attempt to uplift the efficiency level, these reference set

397

)6()6()5()5()3()3()2( 1111 DSDXDSDXDSDXDX ++= (8) where

)2(1 DX is the optimum amount of Input 1 for D2 )3(1 DX is the amount of Input 1 for D3 )5(1 DX is the amount of Input 1 for D5 )6(1 DX is the amount of Input 1 for D6

)3(DS is the dual weight for D3 )5(DS is the dual weight for D5 )6(DS is the dual weight for D6

will be considered as its benchmark. Table II shows the reference set and weights of dual for D2.

TABLE II. REFERENCE SET FOR D2

Department Efficiency Reference Set Weight of Dual

D2 91.96% {D3, D5, D6} {0.6513, 0.0658, 0.1072}

Improvements can be made to any less efficient

department, through either reducing or increasing the amount of inputs or output. Firstly, we will examine the amount input to be reduced in the attempt to produce improvement.

By applying mathematical formula established by [12] to find the optimum amount of Input 1 for D2, the calculation is shown in equation (8).

The optimum amount of Input 2 for D2 can also be calculated by using equation (8) and replacing the amount of Input 1 with Input 2.

Secondly, we will examine the amount of output to be increased in the attempt to produce improvement. The optimum amount of all outputs for D2 can also be calculated using equation (8) and replacing all the inputs with outputs, accordingly.

Table III shows the summary of the current, targeted and the difference values of the inputs and outputs suggested to D2 to increase its efficiency level.

TABLE III. SUMMARY OF THE CURRENT AND TARGETED VALUES OF THE INPUTS AND OUTPUTS FOR D2

Dep

artm

ent

D2

Inputs Outputs

Dep

artm

ent

utili

zatio

n

Prog

ram

of

fere

d

Qua

lity

of

grad

uate

Qua

lity

of

rese

arch

Qua

lity

of

staf

f ser

vice

Current 729 3 40 253 22 Target 675 3 40 253 24 Difference 7.4% 0% 0% 0% 9.1%

The result explains the action that should be taken by D2 in order to improve its performance efficiency level. In a nutshell, the department should reduce its department utilization to only 675, as compared to the current 729. Apart from that, the quality of staff service should be increased from 22 to 24. Other inputs and outputs should remain the same.

IV. CONCLUSION The main objective of the research is to determine the

performance efficiency of six academic departments in a faculty. As the consumption of inputs and the production of outputs are the basic idea in measuring and evaluating the performance efficiency of the departments, DEA technique was employed in this study.

The result shows that, in general, all academic departments in the faculty recorded a high level of efficiency, i.e. above 90%. There are two departments which recorded a perfect efficiency score of 100%, while the least efficient department recorded a score of 91.96%. In order to increase the efficiency score to a satisfactory level, the utilization of the input should be balanced with the production of the output. In line with this, recommendations made are to reduce its Input 1, i.e. department utilization, and to increase its Output 3, i.e. quality of staff.

The results also indicate that the efficient departments are able to serve as benchmarks for the less efficient ones. The findings of the research are supported by other similar studies done in [7], whereby the relative efficiency score recorded in the production of research of Chinese regular universities is above 90%. In fact, the results suggest that the less efficient academic department has the potential to increase its performance level by means of the reference set.

On top of the results obtained, the limitation of DEA should be noted here. Even though DEA provides the suggestions to reach the desired output, it is silent in terms of the way of how the targeted output can be obtained. The DEA reference set which lead to the input reduction should also be accompanied by a prior investigation on the possible organizational factors which may relate to the efficiencies.

For future research, a similar study could be conducted for the whole management of the university. By knowing the performance efficiency of each unit in the university, then proactive measures can be taken to maintain the best practices or to improve the less efficient unit, so as to foster an environment of continuous improvement.

REFERENCES [1] U. Al-Turki and S. Duffaa, “Performance measures for academic

departments,” The International Journal of Educational Management 17/7, 2003, pp. 330-338.

[2] Accreditation board for Engineering and Technology (ABET), 1998, “Engineering criteria 2000: Criteria for Accrediting Programs in Engineering in the United States” 2nd. Ed., Engineering Accreditation Commission, Inc. Baltimore, MD, January, http://www.abet/EAC/eac2000.html

398

[3] Al-Anzi, F., and Alatiqi, I., “An Integrated framework for self-assessment performance analysis at college of engineering and petroleum – Kuwait University”, Proceedings of Symposium on Assessment of Engineering and Technical Education in Saudi Arabia, 1999, p. 43.

[4] Avkiran, N.K., “Investigating technical and scale efficiencies of Australian Universities through data envelopment analysis'', Socio-Economic Planning Sciences , vol. 35, 2001, pp. 57-80.

[5] A. Charnes, W. Cooper and E. Rhodes, “Measuring the efficiency of the Decision Making Units,” European Journal of Operational Research, Vol. 2, 1978, pp. 429-444.

[6] Duffuaa, S.O., Al-Alwani, J.E., and Al-Haddad, A., “Evaluation of engineering and technical education institutions: An integrated approach”, Proceedings of Symposium on assessment of Engineering and Technical Education in Saudi Arabia, 1999, p. 243.

[7] Jill Johnes and Li Yu, “Measuring the research performances of Chinese higher education institutions using data envelopment analysis”, China Economic Review, vol. 19, issue 4, December 2008, pp 679 – 696.

[8] Jill Johnes, “Data envelopment analysis and its application to the measurement of efficiency in higher education”, Economics of Education Review, vol. 25, issue 3, June 2006, pp 273 – 288.

[9]

Wei Meng, Daqun Zhang, Li Qi and Wenbin Liu, “Two-level DEA approaches in research evaluation”, Omega, vol. 36, issue 6, December 2008, pp. 950 – 957.

[10] Tyagi A, Shiv Prasad Yadav A, S. P. Singh B. Efficiency analysis of schools using DEA: A case study of Uttar Pradesh state in India, 2009.

[11] R. Mat Rani and A. R. Saleh, Mengukur kecekapan relatif sekolah – sekolah di Universiti Utara Malaysia, 2002.

[12] S. El-Mahgary and R. Lahdelma, “Data Envelopment Analysis:Visualizing the Result”, European Journal of Operational Research, vol. 85,1995, pp. 700 -710.

399