This article was downloaded by: [Northeastern University]On: 13 October 2014, At: 05:55Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number:1072954 Registered office: Mortimer House, 37-41 Mortimer Street,London W1T 3JH, UK

International Journal ofProduction ResearchPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/tprs20

Assessing computernumerical control machinesusing data envelopmentanalysisShinn SunPublished online: 14 Nov 2010.

To cite this article: Shinn Sun (2002) Assessing computer numerical controlmachines using data envelopment analysis, International Journal of ProductionResearch, 40:9, 2011-2039, DOI: 10.1080/00207540210123634

To link to this article: http://dx.doi.org/10.1080/00207540210123634

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of allthe information (the “Content”) contained in the publications on ourplatform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy,completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views ofthe authors, and are not the views of or endorsed by Taylor & Francis.The accuracy of the Content should not be relied upon and should beindependently verified with primary sources of information. Taylor andFrancis shall not be liable for any losses, actions, claims, proceedings,demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, inrelation to or arising out of the use of the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in any

form to anyone is expressly forbidden. Terms & Conditions of accessand use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

int. j. prod. res., 2002, vol. 40, no. 9, 2011±2039

Assessing computer numerical control machines using data envelopmentanalysis

SHINN SUNy

This paper reports on an application of data envelopment analysis (DEA) toevaluate computer numerical control (CNC) machines in terms of system speci-®cation and cost. The evaluation contributed to a study of advanced manufactur-ing system investments carried out in 2000 by the Taiwanese Combined ServiceForces. The methodology proposed for the evaluation of 21 CNC machines isbased on the combination of the Banker, Charnes, and Cooper (BCC) model(Banker et al. 1984) and cross-e� ciency evaluation (Doyle and Green 1994). Itaims at the identi®cation of a homogeneous set of good systems, by measuring, foreach machine, the pure technical e� ciency through the BCC model. The use ofcross-e� ciency evaluation is to di� erentiate better between good systems and badsystems. These good systems can be further used for the selection of the bestsystems in the decision-making process. The analysis of results might help indus-trial and military buyers as a normative model of system excellence against whichsystem purchasing behaviour could be compared. In addition, it might help pres-ent potential uses for machine manufacturers in competitor analysis and in deter-mining unexplored market niches.

1. Introduction

Advances in information technologies and engineering sciences have substan-

tially increased the number of available Advanced Manufacturing Technologies

(AMTs) and widened their range of performance. Computer numerical control

(CNC) machines, industrial robots, ¯exible manufacturing systems (FMS), auto-mated material handling (AMH) systems, etc, are a few examples of available

options that did not exist or were in limited use 32 years ago. Nowadays, manufac-

turers not only understand that the main advantage of AMTs is labour saving, but

also realize that AMTs provide improved product quality, fast production and

delivery, and increased product ¯exibility. However, because implementing AMT

is very costly and this investment tends to be irreversible, serious consideration is

required before a decision can be made. Evaluation and selection of AMT is acomplex decision making process involving consideration of various issues at the

strategic, tactical and operational levels. Therefore, the investment justi®cation prob-

lems for AMTs have become a major concern of manufacturers all around the globe.

These considerations apply fully to the evaluation and selection of CNC

machines for two reasons. First, comparisons are di� cult to make due to the variety

of machine types available, for example, in the Taiwan market; and secondly, CNC

International Journal of Production Research ISSN 0020±7543 print/ISSN 1366±588X online # 2002 Taylor & Francis Ltd

http://www.tandf.co.uk/journals

DOI: 10.1080/00207540210123634

Revision received December 2001.{ Department of Business Administration and Graduate School of Logistics Management,

National Defence Management College, National Defence University, PO Box 90046-15,Chung-Ho, Taipei 235, Taiwan. e-mail: Shinn@rs590.ndmc.edu.tw

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

machines are usually costly and have many characteristics, which include capability,¯exibility, versatility and reliability.

Over the past several years, the Combined Service Forces (CSF), a militarylogistics agency of the Department of Defense in Taiwan, has considered CNCmachines as cost-e� ective equipment that can be used to perform repetitious, di� -cult, and hazardous tasks with precision in ammunition and shell production. It isestimated that the CSF has invested US$1.2 billion in shop-¯oor automation for itsseven military-owned manufacturing plants in 1990, and the levels of spending haveincreased since then. In selecting the most appropriate automated system for itsmanufacturing factories, the military decision-makers within the CSF have taken afour-phase process for justi®cation of AMT: the planning phase (Phase I), the pre-screening phase (Phase II), the evaluation phase (Phase III), and the selection phase(IV).

Phase I mainly considers the strategic level. The four primary concerns at thislevel are which type of system is presently in operation, what level of actual opera-tion it presently provides, at what level this system has the potential to function, and®nally what the manufacturing plant requires that any new system provide in termsof operations and outputs. This phase requires a rough cut description of the existingsystem and how it can be improved, along with a description of the new system thatwill support the plant’s long term strategies and objectives. In addition, base-lineinformation needs to be determined in terms of both performance and cost data. Theusual criteria considered are investment cost, capacity, product demand, ¯exibility,utilization rate, unit cost and economic risk.

Phase II focuses on the tactical level. The results obtained from Phase I are takeninto the prescreening phase, which aims to evaluate all possible systems through acost/operating risk/performance analysis.

Phase III mainly considers the operational level. Feasible systems selected fromall possible systems at Phase I are evaluated by incorporating technical speci®cationsand acquisition cost into the justi®cation process. The aim of Phase III is to ®ndpotential systems that broadly meet the budget constraints and systems requirementsof a manufacturing factory and to ask vendors of the potential systems to submittheir bids. In the selecting phase, potential vendors are evaluated by taking intoaccount their reputation, sales support, impact on plant capacity, product quality,and bid price. The ®nal choice is then based on the results of vendor assessment andthe lowest bid price.

Following a review of the CSF justi®cation process and interviews with topmanagement, we have identi®ed several justi®cation problems with which the deci-sion-makers are confronted. First, comparisons are hard to make because of thevariety of machine types available in the market. Second, a procedure that assignsweights to various performance measures is essentially an arbitrary process. Theprimary problem associated with arbitrary weights is that they are subjective, andit is often a di� cult task for the decision-makers accurately to assess weightinginformation as the number of performance criteria increase. Third, a decision toolwould be helpful for facilitating the decision-makers’ bargaining with the multiplepotential suppliers using tender prices and tender speci®cations. The decision makingfor AMT selection usually involves bargaining with those suppliers while complyingwith bureaucratic procedures and regulations. A decision method can provide e� ec-tive decision support for bargaining while maintaining accountability. Finally, theCSF has been criticized for not appropriately addressing the issue of `best value for

2012 S. Sun

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

money’ in its purchases by the legislators at the Defense Budgeting Hearing in theLegislative Yuan (a national legislature of the Taiwan government).

In this paper, we develop an alternative method to tackle the problem of how tomeasure one machine against another when each machine can be measured along anumber of technical speci®cations. This is especially useful when no a priori satis-factory weighting scheme exists to combine these speci®cations into an overall ratingfor each machine. We aim to identify a homogeneous set comprising `the best’ orexcellent systems and to discriminate between good and bad systems. The proposedmethodology can also be used as an operational tool for the evaluation of automatedequipment.

The paper describes an application of Data Envelopment Analysis (DEA) toevaluate 21 CNC machines/lathes in terms of system speci®cation and cost at theoperational level. The evaluation contributed to a study of AMT investment carriedout in 2000 by the CSF. Due to the complexities of the AMT evaluation undertakenby the CSF, this paper has not included the whole assessment of CNC machines atthe strategic and tactical levels. The research questions investigated in this studyinclude the following.

(1) Can DEA provide a useful addition to the set of tools and techniques for theevaluation and selection of CNC machines?

(2) Can we obtain useful insights into machine evaluation but re¯ect both theseller’s and buyer’s perspectives?

(3) Are there any shortcomings of DEA when used for the evaluation of CNCmachines?

DEA was initially proposed by Charnes et al. (1978). The use of DEA for theevaluation of CNC machines has some special advantages. The ®rst advantage isthat it provides the decision-makers with e� ective alternative choices by identifying ahomogeneous group of `best-in-class’/e� cient machines. The second advantage isthat it can assist corporate buyers to reconcile a diversity of present and futureuses in one standardized purchase. Other advantage s include, in competitor analysis,determining unexplored market niches and its use as a normative model of machineexcellence as a benchmark for machine purchasing behaviour. More speci®cally, itcan be used easily by decision-makers who may not have the requisite technicalbackground, because it does not need a set of a priori weights for the input andoutput measures from the decision-makers. Readers not familiar with DEA arereferred to Cooper et al. (2000).

The paper is organized as follows. Section 2 provides a literature review of thejusti®cation problems and methods for AMT investments. Section 3 presents a pre-liminary data analysis regarding the selection of input and output measures that canbe used in DEA models. Section 4 introduces the research methodology used in thispaper. Section 5 presents the performance evaluation of CNC machines. Section 6discusses some important implications that emerged from this study. Finally, section7 concludes this paper.

2. Literature reviewOwing to the increased investment in advanced manufacturing technologies,

evaluation, justi®cation and implementation of AMT have been areas of majorconcern and importance for practitioners and researchers. Over the past severaldecades, there has been a considerable amount of research into the development

2013Assessing CNC machines using DEA

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

of models and methodologies for evaluation, justi®cation and implementation ofAMT. Justi®cation tools as they are generally discussed in the literature are ofthree types: (1) accounting methods, (2) analytic methods and (3) strategicapproaches. Recently, a promising approach based on the application of DEA hasbeen recommended as a discrete alternative multiple criteria tool for evaluation andselection of AMT. In this paper, we highlight some important AMT justi®cationproblems, and brie¯y discuss analytic models developed for justi®cation of AMT ingeneral and the use of DEA in particular.

2.1. The justi®cation problems for AMT investmentsThe adoption of AMT is a costly, time consuming, and complex process. This has

largely been in response to many of the investment justi®cation problems, whichhave been addressed extensively over the last two decades. Suresh (1992) presented aclassi®cation of these investment justi®cation problems including:

(1) the high capital costs and risk associated with integrated systems;(2) the myopic approaches to equipment justi®cation emphasizing fast payback

in a climate of short-term-oriented reward systems;(3) the dif®culties in quantifying improvements in lead time and quality, costs

and bene®ts of several types offered by these systems;(4) the dilemma of whether one should explicitly recognize `strategic’ factors

without quanti®cation;(5) the need for extended planning horizons and the need to forecast demand,

price, other parameters, and cash ¯ows over a longer time span;(6) the high rates of obsolescence of processing technologies;(7) the differing nature of operations, based on part family-oriented, software-

driven productions; and(8) the problems relating to ineffective manufacturing strategies.

In addition to these justi®cation problems, Meredith and Suresh (1986) andMeredith and Hill (1987) delineated four considerations often used by corporationsin the economic justi®cation of advanced manufacturing systems. These considera-tions are (1) technical importance, (2) business objective, (3) competitive advantage,and (4) research and development.

Various analytic models have been developed to address some of these issues.The following subsection will provide a brief discussion on the models proposed.

2.2. Models for justi®cation of AMTSeveral researchers have presented comprehensive bibliographies on justi®cation

of AMT in general during the last decade. These references can be found in Falknerand Benhajla (1990), Kolli et al. (1992), Le¯ey (1996) , Proctor and Canada (1992),Sarkis (1992), and Son (1992). Kolli et al. (1992) categorized the economic justi®ca-tion models proposed prior to 1990 into four groups. These groups are: (1) singleobjective deterministic models including net present value, internal rate of return,bene®t/cost ratio, payback period, mathematical programming, and minimal annualrevenue requirement; (2) single objective non-deterministic models including sensi-tivity analysis, decision trees, optimistic/pessimistic analysis, and Monte Carlo simu-lation; (3) multi-objective deterministic models including scoring models, analytichierarchy process (AHP), decision support systems, dynamic programming, goalprogramming, 0-1 multi-objective mathematical programming, outranking

2014 S. Sun

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

approaches and productivity model; and (4) multi-objective non-deterministicmodels including expert systems, fuzzy linguistic methods, game-theoretic models,multi-attribute utility models, and stochastic programming.

In the 1990s, some researchers (Elango and Meinhart 1994, Mohanty andVenkataraman 1993, Sarkis and Liles 1995, Sarkis and Lin 1994) presented strategicjusti®cation frameworks for AMT. Other researchers also used a number of di� erentquantitative models for the AMT justi®cation. These models include: replacementmodels using linear/goal programming (Suresh 1990, 1991, and 1992; and Lot® andSuresh, 1994), multiple criteria decision making models (Agrawal et al. 1991, Kim etal. 1997), outranking methods (Parsaei et al. 1993, Parkan and Wu 1999), risk andsimulation analysis (Kuula 1993), AHP (Albayrakoglu 1996, Kleindorfer andPartovi 1990, Tabucanon et al. 1994), hybrid models such as the combined use ofAHP and goal programming (Stam and Kuula 1991, Suresh and Kaparthi 1992,Myint and Tabucanon 1994).

More recently, the use of DEA for justi®cation of AMT has been addressed byShang and Sueyoshi (1995), Khouja (1995), Schafer and Bradford (1995), Baker andTalluri (1997), Talluri et al. (1997), Sarkis (1997), Sarkis and Talluri (1999), Bragliaand Petroni (1999), Talluri et al. (2000) and Talluri and Yoon (2000). Each of thesepapers has provided important contributions to this area and furthered us in the useof DEA to assess CNC lathes in terms of an e� ciency measure.

There are some key observations on this previous work using DEA.

(1) While the applicability of various DEA models for the AMT justi®cationhas been con®rmed, practical implementations of these proposed method-ologies have not been reported.

(2) Constant returns to scale (CRS) might not be a reasonable assumption forthe AMT problems used in previous works. The CCR (Charnes, Cooper andRhodes 1978) model assumes CRS; thereby, a machine that consumes twiceas much input so should produce double the output is not a reasonableassumption in the evaluation of AMT. The BCC (Banker, Charnes andCooper 1984) model assumes a variable returns to scale (VRS), whichwould imply that a machine that consumes twice as much input might notproduce double the output. VRS is a reasonable assumption for AMT prob-lems because there is no reason to believe that proportionately increasing thecapital cost of a CNC machine would proportionately produce its perform-ance parameters.

(3) It is also possible to determine whether increasing returns to scale (IRS) ordecreasing returns to scale (DRS) as well as constant returns to scale (CRS)are present. However, to date, none of the published research works hasaddressed this issue. Investigating returns to scale is important becausemachine manufacturers can identify their current operating economies ofscale and determine whether their business should be downsizing or expand-ing.

(4) None of the research cited has incorporated organizational aspects, such asbuyers, the organizational buying process and behaviour, and decision cri-teria at various stages of the buying process.

This paper contributes three extensions to the existing research. First, we use realdata concerning the AMT justi®cation process in a real organization, i.e. the CSF,and empirically test the applicability of the proposed model by evaluating 21 CNC

2015Assessing CNC machines using DEA

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

lathes. Secondly, we combine the BCC model and cross-evaluations (Doyle andGreen 1994) to identify a homogeneous set of good systems, by measuring, foreach system, the pure technical e� ciency through the resolution of linear programmeproblems. These good systems can then be used for the selection of AMT systems inthe decision-making process. Finally, we conduct slack analysis to provide informa-tion on the potential improvement for the ine� cient systems and investigate thepotential uses of a DEA analysis.

3. Preliminary data analysisThe complexity of CNC justi®cation includes strategic, tactical, and pecuniary

characteristics. Each of these characteristics may further be categorized into quanti-tative, qualitative, tangible and intangible factors. Factors regarding the CNCmachine investment in the economic justi®cation process are shown in table 1.Several researchers have also addressed some important and basic economic reasonsfor investing in CNC machines. Ste� y et al. (1973) and Smith and Evans (1977)looked at various types of cost savings (e.g. tooling costs). Suresh (1990) identi®edtypes of ¯exibility (product, process, volume, routine and expansion) and strategicadvantages (product development, market development and market penetration,performance improvements, cost savings, acquisition of experience in CNC tech-nologies). Suresh (1992) , Suresh and Kaparthi (1992) and Suresh (1994) addressed

2016 S. Sun

Objective Elements

StrategicMission Match to plan, evolution, economic feasibility, focused factory.Human resources Employee morale, employee development, use of skills, quality of work

life, compensation, absenteeism.Market position Market share, competition assessment, market penetration,

survivability, vulnerability, demand.Organization Control, responsiveness, information managementPublic relations Company image, company prestige, serviceTechnology Scienti®c information yield, technological position, and availability

TacticalFlexibility E� ciency, features, lead times, standardization.Integration Versatility, response to change, batch size, lead times, throughputMaterial Process control, scheduling, part tracking, shop ¯oor control, cost

tracking, expediting, handling, planning.Personnel Skill requirements, training, safety, direct/indirect, human factors.Producibility Consistency, compatibility, feasibility, reliability, capacity, external

interface.

PecuniaryOperation & Operating labour, maintenance labour, direct/indirect costs,Maintenance absenteeism, training, supervision, insurance, overtime, labour

turnover, set-up, maintenance tools/supplies, production rates,documentation, routings, shop ¯oor control.

Plant & Equipment Equipment, start-up, installation, tolling, spare parts, energy, space,safety equipment, hardware development, software development,depreciation, taxes.

Product Design changes, inventory, quality, engineering, sales.

Table 1. Objectives of the CNC investment problem.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

the logic and complexity of justifying ¯exible automation systems based on CNC

technology.

In deciding the input/output measures that we can use to assess CNC lathes we

shall consider technical speci®cations and acquisition costs. This is because the focus

of Phase III in the CSF justi®cation process is to identify a possible set of potential

manufacturers in terms of machine speci®cations and acquisition cost. Other inputs

and outputs not directly relating to technical speci®cations and acquisition cost are

not included in this analysis. A procurement need, identi®ed by the CSF in the

comprehensive study of CNC machine investment (not reported here), was to buy

small-size, vertical CNC lathes as turning centres for the manufacturing plants with a

budget ceiling of unit cost of NT$3 000 000 per unit (1 US$=35.45 NT$ approxi-

mately).

The identi®cation of output measures for evaluating CNC lathes was di� cult.

The performance of a CNC lathe is usually speci®ed by a large number of technical

parameters. Among the most important parameters are work capacity, machine

body, spindle, and tools turret (Thyer 1991, Luggen 1994). Work capacity is meas-

ured by maximum machining diameter (mm) and machining length (mm). The

machine body is measured by rapid traverse rates (m/min) of the X-axis and Z-

axis. The spindle is measured by spindle speed range (rpm). The tools turret is

measured by the tool capacity. Using actual data provided by manufacturers, six

outputs relating mainly to technical speci®cations were chosen by two senior engin-

eers from the CSF. In the next paragraph we will present the reasons for choosing

the six output measures.

A basic CNC turning centre is capable of a wide range of turning operations. The

primary axes of a turning centre are the Z- and X-axes. The Z- and X-axes move-

ment leads to turning centre movements that are controlled by speci®c CNC com-

mands. Rapid traverse rates (m/min) of the X- and Z-axes re¯ect the positioning

capability of a turning centre. According to Thyer (1991), a standard CNC turning

centre has the capability of machining bar work of up to 64 mm diameter by 300 mm

long. The machining diameter and length re¯ect this capability. Spindle speed is the

number of revolutions that a spindle makes in one minute; and it allows a machine to

maintain a constant cutting speed regardless of the part diameter. One problem with

the use of tool turrets is the limited number of tools available. The fewer tools in the

turret that the machine uses, the more time is required to change the tools selected

for use in a particular programme.

In this study, spindle speed range (y1), number of tool capacity (y2), rapid tra-

verse rate of the X-axis (y3); rapid traverse rate of the Z-axis (y4); maximum machin-

ing diameter (y5), and maximum machining length (y6) are treated as outputs to the

DEA models, because they are major measures that indicate the ability of the

systems to perform various tasks. There is only a single input, capital cost (x1),

because from the CSF’s standpoint it represents the investment required to purchase

the system. Table 2 shows the correlation coe� cients.

The following three points are noteworthy.

(1) The input measure is moderately and positively correlated with maximummachining diameter and length. This is because a machine that has a highequipment cost would tend to have high levels of maximum machiningdiameter and machining length.

2017Assessing CNC machines using DEA

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

(2) The rapid traverse range of the X-axis is highly correlated with the rapidtraverse rate of the Z-axis. The correlation coef®cient is 0.92, which is veryhigh. A machine that has a high range of X-axis would tend to have a highrate of Z-axis. This is as we might expect.

(3) There are negative associations among certain output measures. The highestnegative correlation coef®cient is ¡0:785, found between the maximum spin-dle speed and maximum machining diameter. The lowest negative correla-tion coef®cient is ¡0:093, which is not signi®cant, between the spindle speedrange and maximum machining turning length. A machine that has a highspindle speed range would tend to have a small machining turning diameter.

Regression and correlation analysis can empirically infer at least potential input/output relationships, while DEA only presumes such relationships exist. To estimatesuch relationships, one may need to take into account all possible input/outputvariables. We do, however, note that a complete production relationship cannotbe estimated by using only one input and six output variables to compare CNCmachines, since other data for inputs and outputs are not used.

4. Methodology4.1. Sample

The sample consists of 21 CNC lathes that were selected from the results in PhaseII. The type of CNC machine that the CSF considered, was a CNC lathe as a turningcentre for small-size shell production. The speci®ed requirements for a basic turningcentre were (a) the type of the machine should be small and vertical by consideringthe availability of ¯oor space; (b) the diameter of the cutting tool should not be over300 mm, which is the basic work capacity of a CNC lathe; and (c) the equipment costof the machine should not be more than NT$3 millions. Manufacturers of these 21CNC lathes are: Ecoca, Matech, Takisawa, Tong Tai, Yang Iron, Yeong Chin,Victor Taichung, and Far East, which are the major machinery corporations inTaiwan.

We use a production model incorporating one input (equipment cost) and sixoutputs (technical features): the capital cost of a CNC lathe (x1) in New TaiwaneseDollar (NT$), spindle speed range (y1), number of tool capacity (y2), rapid traverserate of the X-axis (y4), rapid traverse rate of the Z-axis (y4), maximum machineturning diameter (y5), and maximum machine turning length (y6). Table 3 representsthe input and output values of the 21 CNC lathes.

There is an operational orientation in the list of inputs and outputs because theevaluation of CNC machines in Phase III focuses on the performance of CNC

2018 S. Sun

x1 y1 y2 y3 y4 y5 y6

Capital cost: x1 1.000 ¡0.314 0.180 ¡0.409 ¡0.306 0.583 0.540Spindle speed range: y1 ¡0.314 1.000 0.104 ¡0.152 ¡0.12 ¡0.785 ¡0.092No. of tool capacity: y2 0.180 0.104 1.000 ¡0.245 ¡0.24 0.014 ¡0.048Rapid traverse rate of X-axis: y3 ¡0.408 ¡0.152 ¡0.245 1.000 0.915 ¡0.017 ¡0.397Rapid traverse rate of Z-axis: y4 ¡0.306 ¡0.254 ¡0.102 0.915 1.000 0.139 ¡0.245Max machining diameter: y5 0.583 ¡0.785 0.0143 ¡0.017 0.139 1.000 0.427Max machining length: y6 0.540 ¡0.093 0.0479 ¡0.397 ¡0.245 0.427 1.000

Table 2. Correlation coe� cients among input and outputs.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

machines in terms of technical and cost factors. These six important technical factorswere selected by the two senior engineers from the CSF as directly capturing theperformance of CNC lathes. Other data (e.g. work approval, ¯oor space) are alsoused in sensitivity analysis (see section 5.6.1). The selection of input and outputvariables in this paper is only relating to the buyer’s concern. We did not considerand include variables relating to the seller’s concern, e.g. what they were worriedabout and their strategies.

4.2. DEA modelsThroughout the study, we use input-oriented DEA models. Input orientation is a

natural choice because the objective of the CSF is to obtain a manufacturingmachine having better performance with least acquisition cost. As noted previously,the performance of CNC machines is proxied in this paper by pure technical andscale e� ciency. In this subsection, we provide a mathematical description of theDEA models of measurement. For other extensions to the basic DEA, models canbe found in Cooper et al. (2000).

Assume the DMU under evaluation has data …x0; y0), and consider the input-oriented CCR DEA mode presented by Charnes et al. (1978), in the primal formula-tion, where xi is an …N £ 1† vector of inputs used by the ith DMU, yi is an …M £ 1†vector of outputs produced by DMU I, and i ˆ 1 . . . I :

TE…x0; y0† ˆ min³;¶

³ …1†

subject to

2019Assessing CNC machines using DEA

Input Output

CNC lathe DMU code x1 y1 y2 y3 y4 y5 y6

YANG ML-15A DMU1 1,200,000 5590 8 24 24 205 350YANG ML-25A DMU2 1,550,000 3465 8 20 20 280 520YCM TC-15 DMU3 1,400,000 5950 12 15 20 250 469VTURN 16 DMU4 1,100,000 5940 12 12 15 230 600FEMCO HL-15 DMU5 1,200,000 5940 12 12 16 150 330FEMCO WNCL-20 DMU6 1,500,000 3465 12 6 12 260 420FEMCO WNCL-30 DMU7 2,600,000 3960 12 12 16 300 625EX-106 DMU8 1,320,000 4950 12 24 30 240 340ECOCA SJ20 DMU9 1,180,000 4480 8 24 24 250 330ECOCA SJ25 DMU10 1,550,000 3950 12 15 20 280 460ECCOA SJ30 DMU11 1,600,000 3450 12 15 20 280 460TOPPER TNL-85A DMU12 1,200,000 3465 8 20 24 264 400TOPPER TNL-100A DMU13 1,350,000 2970 8 20 24 264 400TOPPER TNL-100AL DMU14 1,400,000 2970 12 24 30 300 600TOPPER TNL-85T DMU15 1,350,000 3465 12 30 30 264 350TOPPER TNL-100T DMU16 1,450,000 2970 12 20 24 300 400TOPPER TNL-120T DMU17 1,520,000 2475 12 20 24 300 400ATECH MT-52S DMU18 1,376,000 4752 12 20 24 235 350ATECH MT-52L DMU19 1,440,000 4752 12 20 24 235 600ATECH MT-75S DMU20 1,824,000 3790 10 12 20 300 530ATECH MT-75L DMU21 1,920,000 3790 10 12 20 300 1030

Table 3. Input and output values of the 21 CNC lathes.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

³x0 ¡ X¶ ˆ 0

¡ y0 ‡ Y¶ ¶ 0

¶ ¶ 0;

where X is N £ I input matrix with columns xi, Y is an M £ I output matrix withcolumns yi, and ¶ is an i £ 1 intensity vector. The optimal value of ³ provides atechnical e� ciency measure of the DMU under evaluation.

Banker et al. (1984) proposed a DEA model referred to the BCC model bygeneralizing the CCR formulation to allow variable returns to scale. The input-oriented BCC DEA model computes a pure technical e� ciency measure by intro-ducing an additional restriction to the CCR DEA model: eT¶ ˆ 1, where eT is anI £ 1 row vector of ones. Pure technical e� ciency can be computed as a solution tothe following linear programming (LP) problem:

PTR…x0; y0† ˆ min³;¶;¼

³ …2†

subject to

³¼x0 ¡ X¶ ˆ 0

¡ y0 ‡ Y¶ ¶ 0

eT¶ ˆ 1

0 < ¼ < 1

¶ ¶ 0:

Then, scale e� ciency for DMU I is obtained as:

SE i ˆ TEi;CRS=PTEi;VRS: …3†

It represents the proportion of inputs that can be further reduced after pure technicaline� ciency is eliminated if scale adjustments are possible. It has a value of less thanor equal to one. If machine I has a value equal to one, it means that the manufacturerof machine I is operating at the constant returns to scale region. If SE i is less thanone, then the manufacturer of machine I is scale ine� cient and there is a potentialinput saving through the adjustment of its operational scale. Depending on the scaleine� ciency for the ith machine, the manufacturer of this machine should be eitherdownsizing or expanding relative to its current operating scale.

To determine the current economy of scale that a manufacturer is experiencing,an e� ciency index must be computed when the technology exhibits non-increasingreturns to scale (NIRS). This can be done by relaxing the constraint on the weightvariables in (2) to be less than or equal to one. The solution to the LP problem formachine I when eT¶ ˆ 1 is replaced by eT¶ ¿ 1 is denoted by E i;NS. Again, the LPproblem is solved for each machine. TE i;NRS is then compared to REi;VRS . IfTEi;NRS ˆ TEi;VRS, then the manufacturer of the scale ine� cient machine I is experi-encing decreasing returns to scale and would be bene®ted by downsizing. On theother hand, if TE i;NRS < TE i;VRS , then the manufacturer of the scale ine� cientmachine I is operating increasing returns to scale and should expand its production.For a detailed discussion on returns to scale (RTS), see Banker and Thrall (1992).

2020 S. Sun

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

5. Results5.1. E� ciency analysis

Table 4 presents a summary of various e� ciency measures for the 21 CNC lathes.

It shows that (1) nine of the 21 machines were relatively ine� cient under CRS with a

mean e� ciency score 90.4; (2) nine of the 21 machines were technically ine� cient

under VRS with a mean e� ciency score 95.2; (3) 12 of the 21 lathes were scale

ine� cient with a mean e� ciency score 95.1; and (4) the returns to scale categories

for IRS, CRS, DRS are 0, 14, and 7 machines, respectively.

Several points can be made.

(1) The ef®ciency score obtained from the BCC/VRS model re¯ects a certainmanufacturer’s current scale of machine manufacturing and represents theability of management to transform inputs to produce outputs. Thus,YANG ML-15A (Yang Iron), YCM-TC-15 (Yeong Chin), VTURN 16(Victor Taichung), EX-106 (Takisawa), ECOCA SJ20 (Ecoca), FEMCOWNCL-30 (Far East), TOPPER TNL-85A, -85T and -100AL (Tong Tai)and ATECH MT-52L, -75S, and -75L were identi®ed as `best performance’machines.

(2) The mean scale ef®ciency of 95.1 in table 4 suggests further potential inputsavings of 4.9 if it is possible for a manufacturer to produce a CNC lathe atthe constant returns to scale technology. Investigating the distribution of

2021Assessing CNC machines using DEA

E� ciency measuresDMU

CNC lathe code Crste Vrste scale RTS

YANG ML-15A DMU1 1.000 1.000 1.000 crsYANG ML-25A DMU2 0.835 0.838 0.964 crsYCM TC-15 DMU3 0.875 1.000 0.875 drsVTURN 16 DMU4 1.000 1.000 1.000 crsFEMCO HL-15 DMU5 0.935 0.966 0.968 drsFEMCO WNCL-20 DMU6 0.818 0.819 0.999 crsFEMCO WNCL-30 DMU7 0.539 1.000 0.539 drsEX-106 DMU8 1.000 1.000 1.000 crsECOCA SJ20 DMU9 1.000 1.000 1.000 crsECOCA SJ25 DMU10 0.846 0.869 0.974 drsECCOA SJ30 DMU11 0.819 0.821 0.998 crsTOPPER TNL-85A DMU12 1.000 1.000 1.000 crsTOPPER TNL-100A DMU13 0.889 0.889 1.000 crsTOPPER TNL-100AL DMU14 1.000 1.000 1.000 crsTOPPER TNL-85T DMU15 1.000 1.000 1.000 crsTOPPER TNL-100T DMU16 0.963 0.966 0.997 crsTOPPER TNL-120T DMU17 0.918 0.921 0.997 crsATECH MT-52S DMU18 0.898 0.898 1.000 crsATECH MT-52L DMU19 0.914 1.000 0.914 drsATECH MT-75S DMU20 0.757 1.000 0.757 drsATECH MT-75L DMU21 0.984 1.000 0.984 drs

Mean 0.904 0.952 0.951

Note: crste = technical e� ciency from CRS DEA; vrste = technical e� ciency from VRSDEA; scale = scale e� ciency = crste/vrste.

Table 4. E� ciency of the 21 CNC lathes.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

scale in table 4 might imply that the manufacturers of these 14 machinesalready practice economies of scale at the appropriate level. Some seven outof the 21 machines are experiencing decreasing returns to scale. It mightsuggest that the manufacturers of these seven machines could reduce theirsize of economies of scale toward ef®ciency by adjusting the current inputand output levels.

5.2. Review of peer referencesAn inspection was next made of how frequently each e� cient machine was used

as a comparator or `e� cient peer’ for ine� cient machines. The purpose of thisinspection was to identify an exemplar of good performance according to thenumber of times e� cient lathes appeared in a reference set. A reference set stakesout the e� ciency frontier that any particular machine is aiming for. The referencesets and their frequencies for each machine are given in table 5.

From the manufacturer’s point of view, the reference set indicates who are thebest of the competition in terms of their machine speci®cations. It is clear that somemachines achieve e� ciency by occupying a `technical niche’ in the market. That is,they aim for superlative performance in any feature A, while only providing asatisfactory level when it comes to features B and C.

How do we distinguish among the 100% e� cient sets between the niche and thebroad player? The references set in table 5 contains this information. According toDoyle and Green (1991) , niche players will seldom appear in the reference sets of

2022 S. Sun

BCC/VRS ModelDMU

code Reference set Frequency

DMU1* 1 0DMU2 4 12 14 0DMU3 3 2DMU4 4 6DMU5 3 4 8 0DMU6 4 14 0DMU7* 7 0DMU8 8 2DMU9* 9 0DMU10 3 4 14 0DMU11 4 14 0DMU12 12 2DMU13 12 0DMU14 14 6DMU15 15 1DMU16 14 0DMU17 14 0DMU18 4 8 15 0DMU19* 19 0DMU20* 20 0DMU21* 21 0

Note: A DMU with a * is a niche player.

Table 5. Reference set for the 21 CNC lathes.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

other machines because, almost by de®nition, there can be few competitors in anyone niche. Lathes with a broad scope will appear in many others’ reference set.

Consequently, if we add the number of times each 100% e� cient lathe appears in

others’ reference sets we should get an order of machines from broad to niche. This

has been done under the `Frequency’ column of table 5. Several observations can be

made.

(1) Six of the 12 ef®cient lathes with an asterisk mark are the niche players sincetheir frequency to other machines is zero. No other machine has them intheir reference sets. It is extraordinary to think that these six niche playershave so few competitors in this part of the market.

(2) Surprisingly, it was found that some expensive CNC lathes, such as FEMCOWNCL-30, ATECHMT-75S and 75L, were ef®cient while other expensivemachines tend to be less ef®cient. Manufacturers of these three expensivemachines can obtain economies of scale and keep their prices high. Becauseof high quality outputs, they can compete in a crowded part of the market,which achieves 100% ef®ciency.

(3) There is no evidence to support a claim that cheaper machines tend to be lessef®cient. The correlation between cost and ef®ciency is -0.04, which is notsigni®cant. Presumably, more cheap lathes are sold than expensive ones;manufacturers of cheap lathes can therefore obtain economies of scale tokeep their prices low. Given a large turnover, they can survive on smallermargins.

5.3. Slack analysis

A non-zero slack analysis was used to cross-check the results of e� ciency for

ine� cient machines. Non-zero slack analysis can identify marginal contributions in

e� ciency ratings with an additional increase in speci®c output amounts or with an

additional decrease in speci®c input amounts. Table 6 represents results of the slackanalysis.

2023Assessing CNC machines using DEA

Input OutputDMU

Code x1 y1 y2 y3 y4 y5 y6

DMU2 0 0 0 1.315 6.301 0 11.507DMU5 0 0 2.630 0 0.606 83.902 244.061DMU6 0 1202.143 0 0 11.143 0 180DMU10 0 0 0 0 5.506 0 119.947DMU11 0 368.571 0 0 5.571 0 140DMU13 0 0 0 0 0 0 0DMU16 0 0 0 0 4 0 200DMU17 0 495 0 0 4 0 200DMU18 0 396 0 0 0 4.2 95.333

Number of 0 4 1 1 7 2 8DMUswith slacksMean 0 615.429 2.630 1.315 5.304 62.951 148.856

Table 6. Slacks for each input /output for the ine� cient DMUs.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

The input slack of zero for each ine� cient machine shows that those machineshave not excessively used their input amounts in determining the e� ciency score.Amongst the output items, the highest number of non-zero slacks is eight forthe maximum machining length. Holding the level of the manufacturer’s practiceconstant, on average, four DMUs could have increased the spindle speed range by615.429 rpm; one DMU could increase the tool capacity by 2.630; one DMU couldincrease the rapid traverse rate of the X-axis by 1.315 m/min; seven DMUscould increase the rapid traverse rate of the Z-axis by 5.304 m/min; twoDMUs could increase the maximum machining diameter by 62.951 mm; and eightDMUs could increase the maximum machining length by 148.856 mm. These esti-mated increases in the outputs are additional to any estimated reductions in the inputif a unit were to achieve 100% e� ciency under an input orientation.

5.4. Review of ine� cient units5.4.1. Contrasting ine� cient lathes with their e� cient peers

A manufacturer of an ine� cient lathe who wishes to see how well the machineperforms with respect to its competitors could ®nd DEA very useful. A good apprai-sal of the performance of each ine� cient lathe can be gained when its input±outputlevels are compared with those of its e� cient peer machines identi®ed in table 4. Thecomparisons were made using tables such as table 7.

Table 7 relates to FEMCO WNCL-20/DMU6. The column headed `FEMCOWNCL-20’ shows its `observed’ input-output levels. The input and output values forDMU6 have been scaled to equal 100%. The input-output levels under VTURN16are expressed as a percentage of DMU6’s values. The values under TOPPER TNL-100AL have been scaled in a similar manner. The scaling of the peer input±output inthis way makes all the input levels no higher than the corresponding level of theine� cient lathe. This makes it easy to compare DUM6 with its peers as we can focuson output levels only.

If DMU6 is deemed to have equivalent performance to that of its e� cient peers,its output levels must be at least as good as those of its e� cient peers. However, oneof the output levels under DMU6 is better than its peers while none of the rest of itsoutput levels is higher than the corresponding levels of its e� cient peers. Tables suchas table 7 can be used to review the performance of all ine� cient lathes.

2024 S. Sun

Ine� cient unit Scaled e� cient set

FEMCO WNCL-20 VTURN 16 TOPPER TNL-100AL

Capital cost (NT$): x1 100 51.16 48.84Spindle speed range (rpm): y1 6.98 72.73 27.27No. of tool position (no.): y2 10.959 57.14 42.86Rapid traverse rate of X-axis

(m/min): y3 2.397 40 60.00Rapid traverse rate of Z-axis

(m/min): y4 4.795 40 60.00Max machining diameter (mm):

y5 89.041 50.55 49.45Max machining length (mm):

y6 4.888 57.14 42.86

Table 7. E� cient peers for FEMCO WNCL-20 (e� ciency 81.9%).

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

5.4.2. Targets for ine� cient lathes

The solution of the DEA model yields target input and output levels for ine� -

cient CNC lathes (see Charnes et al. 1978). Information on target levels is given in

table 8. This information can be used to provide manufacturers of CNC lathes with

assistance to improve performance of their products.

The top columns show the amounts of input and output that an ine� cient lathe

should be using or producing to become e� cient while the lower columns show the

possible improvement as a percentage of the current level.

Three points are worthy of note here.

(1) There are numerous ways to become ef®cient and the DEA slacks suggestone of them. If a vendor wants to compete on a smaller number of factors,or a buyer wants a certain speci®c characteristic, a DMU could try to focuson that factor. Alternatively, a DMU could focus on factors that the marketviews as important (DEA does not really tell us thisÐthis is a supplier orcustomer preference issue).

(2) It may not be possible for a manufacturer selectively to improve the per-formances of a speci®c factor on an inef®cient machine owing to high costand/or technical complexity.

(3) Whether machine manufacturers would like to improve the performance ofinef®cient lathes by adjusting the target improvement levels depends on themanufacturers’ willingness.

2025Assessing CNC machines using DEA

TargetDMU

code x1 y1 y2 y3 y4 y5 y6

2 1298630 3465 10.630 21.315 26.301 280 531.5075 1158667 5940 12 12.606 16 233.902 574.0616 1228571 4667.143 12 17.143 21.429 260 600

10 1347088 3950 12 20.506 25.824 280 579.94711 1314286 3818.571 12 20.571 25.714 280 60013 1200000 3465 8 20 24 264 40016 1400000 2970 12 24 30 300 60017 1400000 2970 12 24 30 300 60018 1236000 5148 12 20 24 235 600

Potential improvement (%)DMU

code x1 y1 y2 y3 y4 y5 y6

2 ¡16.22 0 32.88 6.58 31.51 0 2.215 ¡3.44 0 0 5.05 0 55.93 73.966 ¡18.10 34.69 0 185.71 78.57 0 42.86

10 ¡13.09 0 0 36.71 29.12 0 26.0811 ¡17.86 10.68 0 37.14 28.57 0 30.4313 ¡11.11 16.67 0 0 0 0 016 ¡3.45 0 0 20.00 25.00 0 50.0017 ¡7.89 20.00 0 20.00 25.00 0 50.0018 ¡10.17 8.33 0 0 0 1.79 27.24

Table 8. Targets and potential improvements for ine� cient CNC lathes.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

5.5. Cross-e� ciency analysisThe use of cross e� ciency in DEA has been addressed by Sexton et al. (1986),

Doyle and Green (1994), Baker and Talluri (1997) and Sarkis and Talluri (1999).Sexton et al. (1986) ®rst introduced the concept of cross-e � ciency in DEA.Mathematically, the cross-e� ciency of DMU k can be formulated as:

MaxX

s

vsysk …4†

subject to

X

j

ujxjk ˆ 1

X

s

vsysi ¡X

j

ujxji ¿ 0 8i

vsuj ¶ 0 8s and j;

where s ˆ 1 through p; j ˆ 1 through m; i ˆ 1 through n; xji ˆ amount of input jused by DMU i; ysi ˆ amount of output s produced by DMU i; uj ˆ weight attachedto input j; vs ˆ weight attached to output s; m ˆ number of inputs; and p ˆ numberof outputs; and n ˆ the number of DMUs. A matrix of cross-e� ciencies (CEM) canbe obtained by repeating LP model (4) for all DMUs.

Doyle and Green (1994) note that input/output weights (optimal weights)obtained from formulation (4) may be not unique. This condition occurs if multipleoptimum solutions exist. Thus, DMU k’s evaluation of other DMUs may depend onwhich of the alternative solutions are found ®rst by the solution process. Because thechoice of the weights in¯uences the CEM for the other DMUs, its casualness mayimply distorted judgements. Therefore, they propose aggressive and benevolent for-mulations to solve this ambiguity. These formulations not only maximize the e� -ciency of the target DMU, but they also take a second goal into account. This secondgoal, in the case of aggressive formulation, is to minimize the e� ciency of the com-posite DMU constructed from the other n ¡ 1 DMUs. The outputs and inputs of acomposite DMU are obtained by summing the corresponding output and inputs ofall the other DMUs except the target DMU. The weights obtained from this for-mulation make the e� ciency of the target DMU the best that it can be, and all otherDMUs worst. Thus, the CEM evaluated from these weights becomes more mean-ingful. The aggressive formulation proposed by Doyle and Green (1994) is generallyused when relative dominance among the DMUs is to be identi®ed. The formulationis:

MinX

s

vs

X

i 6ˆk

ysi

Á !…5†

subject to

X

j

uj

X

i 6ˆk

xji

Á !ˆ 1

2026 S. Sun

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

X

s

vv; ysi ¡X

j

ujxji ¿ 0

X

s

vv; ysk ¡ ³kk

X

j

xjk ˆ 0

vs; uj ¶ 0 8s and j;

where DMU k is the target DMU;P

s…vs

Pi 6ˆk yki† is the weighted output of com-

posite DMU;P

j…uj

Pi 6ˆk xji† is the weighted input of composite DMU; ³kk is the

e� ciency of DMU k obtained from formulation (4).The benevolent formulation uses the same set of constraints except that the

e� ciency of the composite DMU is maximized. In this paper, we utilize the aggres-sive formulation since we are interested in identifying dominance amongst the 21machines.

Doyle and Green (1994) also present a `Maverick index’ as an e� ective way ofmeasuring the false positiveness of DMUs. The maverick index can be de®ned as thepercentage relative di� erence between cross-e � ciency and simple e� ciency. That is:

Mk ˆ Ekk ¡ ek

ek

£ 100; …6†

where Mk is the maverick index for the kth DMU, Ekk and ek are, respectively, itssimple e� ciency and its cross-e� ciency. An e� cient DMU with a large maverickindex value is likely to be e� cient on only a small number of measures.

Another way is to build a `false positive index’ (FPI) (Baker and Talluri 1997),which shows the percentage increment in e� ciency that a DMU achieves whenmoving from peer-appraisal to self-appraisal . Peer-appraisal for a DMU is how itis rated by other DMUs and its mean score is a good measure for this. Self-appraisalis measured as follows:

FPIk ˆ ³kk ¡X

i

³=n

Á !Á !¿ X

i

³ik=n

Á !

; …7†

where ³kk is the simple of DMU k, (P

i ³ik=n† is the mean score of DMU k obtainedfrom the CEM. A low FPI for a DMU indicates that it bene®ted the least whenmoving from peer-appraisal to self-appraisal.

In our application, the cross-e� ciency aggressive method, maverick index-aggressive method, and FPI were all used to discriminate among these e� cientmachines. The CEM evaluation from the optimal DEA weights is shown in table9. Table 10 reports the values of the mean scores for self-appraisal, the maverickindex, and the FPIs for the 21 lathes used in the analysis.

It can be seen from table 9 that lathes 4, 9, 14, 12, 1, 15 and 8 have several highcross-e� ciency values and lathe 7 has the lowest cross-e� ciency values. The columnmeans of the CEM can be used to di� erentiate e� ectively between overall e� cientlathes and possible `false positive’ candidates. Lathe 7, which was simple and e� -cient, exhibited the lowest mean score of 0.42. This lathe is far from being a goodoverall candidate, even though it outperforms the other e� cient ones on its DEAscore. The other e� cient lathes 1, 3, 4, 8, 9, 12, 14, 15, 20 and 21 exhibited high meanscores of 0.86, 0.78, 0.94, 0.86, 0.90, 0.87, 0.87, 0.86, 0.60, 0.61, respectively. Lathe 4with a mean score of 0.94 is rated the best by the other lathes.

2027Assessing CNC machines using DEA

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

2028 S. Sun

I/J

L1

L2

L3

L4

L5

L6

L7

L8

L9

L10

L11

L12

L13

L14

L15

L16

L17

L18

L19

L20

L21

L1

1.00

0.4

90.8

11.0

00.9

20.4

30.

30

0.7

90.8

10.

51

0.4

40.6

30.

500.5

00.

62

0.4

60.

39

0.7

10.6

80.4

10.3

9L

20.

79

0.8

40.8

21.0

00.5

90.7

80.

54

0.8

30.9

60.

82

0.8

01.0

00.

891.0

00.

89

0.9

30.

89

0.7

80.7

80.7

50.7

8L

31.

00

0.7

10.8

71.0

00.8

20.6

20.

43

1.0

01.0

00.

74

0.6

90.9

50.

820.9

20.

94

0.8

20.

76

0.8

70.8

30.6

30.6

0L

40.

92

0.4

10.7

91.0

00.9

20.4

30.

28

0.6

90.7

00.

47

0.4

00.5

30.

410.3

90.

48

0.3

80.

30

0.6

40.6

10.3

80.3

7L

51.

00

0.5

80.8

61.0

00.9

30.5

40.

36

1.0

00.9

10.

66

0.6

10.8

10.

690.8

20.

88

0.7

00.

64

0.8

60.8

20.5

30.5

0L

60.

79

0.8

20.8

51.0

00.6

40.8

20.

54

0.8

70.9

70.

85

0.8

21.0

00.

891.0

00.

92

0.9

60.

92

0.8

10.7

80.7

50.7

2L

70.

78

0.8

30.8

31.0

00.5

90.7

90.

54

0.8

20.9

50.

83

0.8

1.0

00.

891.0

00.

88

0.9

30.

89

0.7

80.7

80.7

60.7

8L

80.

88

0.5

70.6

30.6

00.5

90.3

50.

27

1.0

00.8

90.

57

0.5

50.8

80.

780.9

40.

98

0.7

30.

69

0.7

70.7

30.4

80.4

6L

91.

00

0.6

70.6

70.7

50.6

30.3

70.

31

0.9

01.0

00.

57

0.5

40.8

50.

740.8

31.

00

0.7

10.

67

0.7

60.7

30.4

40.4

2L

10

0.92

0.4

10.7

91.0

00.9

20.4

30.

28

0.6

90.7

00.

47

0.4

00.5

30.

410.3

90.

48

0.3

80.

30

0.6

40.6

10.3

80.3

7L

11

0.79

0.8

20.8

51.0

00.6

40.8

20.

54

0.8

70.9

70.

85

0.8

21.0

00.

891.0

00.

92

0.9

60.

92

0.8

10.7

80.7

50.7

2L

12

0.87

0.7

80.7

70.8

60.5

90.6

50.

45

0.9

21.0

00.

76

0.7

41.0

00.

891.0

00.

98

0.9

10.

86

0.8

10.7

70.6

70.6

4L

13

0.80

0.8

20.8

51.0

00.6

40.8

10.

53

0.8

70.9

70.

84

0.8

21.0

00.

891.0

00.

93

0.9

60.

92

0.8

20.7

80.7

50.7

1L

14

0.96

0.7

40.8

51.0

00.7

80.6

40.

45

1.0

01.0

00.

76

0.7

30.9

60.

851.0

01.

00

0.8

70.

82

0.8

70.8

60.6

40.6

6L

15

0.90

0.5

80.4

80.4

90.4

50.1

80.

21

0.8

20.9

20.

44

0.4

20.7

50.

670.7

71.

00

0.6

20.

59

0.6

50.6

30.3

00.2

8L

16

0.80

0.8

20.8

51.0

00.6

40.8

10.

53

0.8

70.9

70.

84

0.8

21.0

00.

891.0

00.

93

0.9

60.

92

0.8

20.7

80.7

50.7

1L

17

0.80

0.8

20.8

51.0

00.6

40.8

10.

53

0.8

70.9

70.

84

0.8

21.0

00.

891.0

00.

93

0.9

60.

92

0.8

20.7

80.7

50.7

1L

18

0.80

0.5

80.8

31.0

00.9

30.6

90.

43

1.0

00.8

20.

75

0.7

30.7

90.

700.9

41.

00

0.8

50.

81

0.9

00.8

60.5

50.5

2L

19

1.00

0.7

80.7

51.0

00.6

80.4

60.

42

0.8

90.9

70.

65

0.6

20.9

00.

791.0

00.

99

0.7

40.

70

0.7

70.9

10.5

40.7

8L

20

0.78

0.8

30.8

31.0

00.5

90.7

90.

54

0.8

20.9

50.

83

0.8

01.0

00.

891.0

00.

88

0.9

30.

89

0.7

80.7

80.7

60.7

8L

21

0.53

0.6

20.6

11.0

00.5

00.5

10.

44

0.4

70.5

10.

54

0.5

30.6

10.

540.7

90.

48

0.5

10.

48

0.4

70.7

60.5

30.9

8

Mea

n0.

86

0.6

90.7

80.9

40.7

00.6

10.

42

0.8

60.9

00.

70

0.6

60.8

70.

760.8

70.

86

0.7

80.

73

0.7

70.7

60.6

00.6

1

Fo

rex

amp

le,

0.4

9(t

he

valu

ein

the

®rs

tro

wan

dse

cond

colu

mn)

rep

rese

nts

the

cro

ss¡

e�ci

ency

score

of

lath

e2

wit

hth

eop

tim

alw

eigh

tso

fla

the

1.

Tab

le9.

Am

atr

ixo

fcr

oss

-e�

cien

cy.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

As we can see from table 10, some of the e� cient lathes are in fact highly `false

positive’. E� cient lathes 21, 20, 7 and 19 exhibited high false positive indices of

60.71%, 27.02%, 26.9%, respectively. Lathe 4 had the least FPI of 6.53% and

also the highest mean score. A low FPI indicates that the lathe obtains little bene®t

moving from peer-appraisal to self-appraisal.Despite the fact that di� erent values of cross-e � ciency maverick indices are

obtained, the three formulations lead to the same conclusions, especially for e� cient

lathes. Lathe 4 is the top rank in three cases, followed by lathe 9. With the aggressive

formulation for lathe 7, the maverick index is 138.1% (more than double that of any

other lathe) with a cross-e� ciency value of 0.42, lower than most of the lathes with

e� ciency < 1. It is plausible that such an apparently e� cient lathe is best at some

things (maximum machining diameter and length) but is very poorly rated withrespect to other attributes.

On the other hand, cross-e� ciency appears to be the superiority of lathes 4 and 9.

Lathe 4 has the edge over lathe 9 due to its false positive index 6.53% (versus

10.81%) and its cross-e� ciency value of 0.94 (versus 0.90). Thus, lathe 4 is a good

overall candidate that performs well over most dimensions. It can be seen that this

methodology allows decision-makers to rank the lathes based on their overall per-formance. The optimal choice may not necessarily be the one with the highest

column mean since there may be other intangible evaluative criteria in the vendor

assessment, such as vendor reputation, sales support, etc. The value of this analysis is

to assist decision-makers in not selecting a `false positive’ lathe or considering lathes

(with an e� ciency score of less than 1) as being probable candidates for the ®nal

choice.

2029Assessing CNC machines using DEA

Lathe CCR BCC Cross-e� ciency Maverick FPINo. e� ciency e� ciency mean Index (%) (%)

4 1 1 0.94 6.4 6.539 1 1 0.90 21.3 10.813 0.87 1 0.78 16.3 11.65

14 1 1 0.87 16.3 14.8412 1 1 0.87 11.1 15.361 1 1 0.86 38.6 15.77

15 1 1 0.86 16.3 16.068 1 1 0.86 31.6 16.68

18 0.9 0.9 0.77 28.2 17.0113 0.89 0.89 0.76 14.9 17.4519 0.91 1 0.76 14.9 19.562 0.84 0.84 0.69 16.9 20.98

10 0.85 0.87 0.70 21.7 21.6511 0.82 0.82 0.66 26.0 23.8016 0.96 0.97 0.78 17.1 24.1717 0.92 0.92 0.73 24.4 26.257 0.54 1 0.42 138.1 26.90

20 0.76 1 0.60 66.7 27.025 0.93 0.97 0.70 34.4 34.036 0.82 0.82 0.61 24.2 34.77

21 0.98 1 0.61 63.9 60.71

Table 10. Values of simple e� ciency, cross-e� ciency, maverick index and FPI.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

5.6. Sensitivity analysisIn order to gain further insights into the initial results, we decided to test the

sensitivity of results to changes in input-output speci®cation and weight restrictions.As indicated by Sexton (1986) and Nunamaker (1985), the relative e� ciency

achieved by each DMU can be sensitive to the number of inputs and outputs. Ingeneral, the more input and output variables that are included in the model, thehigher will be the number of DMUs with an e� ciency score equal to unity. CCR andBCC models place no constraints on the weight attributed to each input and eachoutput in the multiplier problem, thus allowing absolute weight ¯exibility. This canresult in identifying a DMU with an extreme weighting scheme as being e� cient.Outlier units will tend to be classi®ed as technically e� cient and zero weights will beassigned to most of their inputs/outputs. This represents a contradiction in itselfbecause, if such inputs/outputs were not important, they would not be included inthe analysis. Absolute weight ¯exibility can result in an overestimation of technicale� ciency. Therefore, it sometimes may be appropriate to place weight restrictions tore¯ect to the priorities of the decision-maker.

In addition, bootstrapping (see Efron 1979) can be used to analyse the sensitivityof e� ciency scores to sampling variation. Bootstrapping is based on the idea ofrepeatedly simulating the data generating process (DGP), usually through resam-pling, and applying the original estimator to each simulated sample so that resultingestimations mimic the sampling distribution of the original estimator. In principle,this can be done for any statistics (estimator) de®ned on the data, provided theunderlying DGP is properly simulated. According to Simar and Wilson (1998), theprimary di� culty in applying bootstrap methods in complex situations, such as thecase of non-parametric frontier estimation, lies in stimulating the DGP. In the caseof non-parametric frontier estimation, one must ®rst clearly de®ne a model of theDGP. If the DGP is not speci®ed a priori, we cannot know whether the bootstrapmimics the sampling distribution of the estimators of interest, or some other distri-bution. A general methodology of bootstrap sampling in non-parametric frontiermodels is given in Simar and Wilson (1998).

5.6.1. Changes in input-outpu t speci®cationIt was decided to run a series of six DEAs, working up from a simple analysis

involving just one input and six outputs to two inputs and eight outputs, in order toverify whether the performances of CNC lathes were balanced with respect to thevarious attributes. Table 11 shows six possible combinations of input and outputsfactors. In addition to the six outputs and one input previously used in the originalDEA model, two further outputs (worker approval, y7; maximum spindle speed, y8)and one input (¯oor space, x2) were selected for running the six alternative DEAformulations. These additional input/output measures are not intended to be exhaus-tive, but are some general measures suggested by Thyer (1991), Luggen (1994),Shang and Sueyoshi (1995) , and Talluri et al. (2000).

Worker approval is the ease of operation of the lathe and was included as aqualitative output, which is measured on an ordinal scale of 1 to 5, where 1 is a leastpreferable score. Maximum spindle speed (rpm) is important to the control of rota-tional movements because it a� ects the roundness of the components produced.Floor space is the ¯oor space requirements of each speci®c machine, measured insquare metres (m2). Minimum ¯oor space utilization can reduce the actual storagerequirements of these expensive CNC machines. Table 12 shows the results of all

2030 S. Sun

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

DEAs undertaken. These are labelled Model I to Model VI, where model I is the

original DEA model.

As can be seen from table 12, most lathes appeared on the e� ciency frontier

under Models IV, V and VI. In addition to the previous 12 e� cient lathes originally

identi®ed by running Model I, four other e� cient lathes were introduced into the

e� ciency frontier, namely lathes 5, 10, 16 and 17. This impractical outcome could be

attributed to the fact that a large number of factors (in particular, both on the inputand output sides) was used in the analysis of a small number of lathes. This may not

seem very surprising. Some of the factors fail to discriminate e� ciency di� erences

among lathes rather than highlight these di� erences. Consequently, one could elim-

inate some irrelevant factors and isolate the most relevant factors for the DEA

analysis.

2031Assessing CNC machines using DEA

Input Output

Model x1 x2 y1 y2 y3 y4 y5 y6 y7 y8

I £ £ £ £ £ £ £II £ £ £ £ £ £ £ £III £ £ £ £ £ £ £ £ £IV £ £ £ £ £ £ £ £V £ £ £ £ £ £ £ £ £VI £ £ £ £ £ £ £ £ £ £

Table 11. Combinations of input and output factors.

Lathe ModelNo. I II III IV V VI

1 100 100 100 100 100 1002 83.78 83.78 84.75 83.78 84.75 84.753 100 100 100 100 100 1004 100 100 100 100 100 1005 96.56 96.56 96.67 100 100 1006 81.90 81.90 81.90 81.90 81.90 81.907 100 100 100 100 100 1008 100 100 100 100 100 1009 100 100 100 100 100 100

10 86.91 86.91 87.10 100 100 10011 82.14 82.14 82.14 82.14 82.14 82.1412 100 100 100 100 100 10013 90.90 90.90 90.90 91.09 91.09 91.0914 100 100 100 100 100 10015 100 100 100 100 100 10016 96.56 96.56 96.55 100 100 10017 92.11 92.91 92.11 100 100 10018 89.83 89.83 89.83 89.83 89.83 89.8319 100 100 100 100 100 10020 100 100 100 100 100 10021 100 100 100 100 100 100

Table 12. DEA scores for 21 CNC lathes.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

5.6.2. Changes with weight restrictionsIt is not easy to decide how weights are to be constrained. For example, what

weights should the decision-maker provide for rapid traverse rates (m/min) of the X-and Z-axes? Practically, weights are di� cult to interpret. Braglia and Petroni (1999)indicate that one of the main advantages of DEA is objectivity. This property isusually limited when weight constraints are added. On the other hand, a subjectiveconstraints selection may lead to the same inconvenience. Dyson and Thanassoulis(1988) propose a possible alternative that restricts the weights in such a way thateach weight is decreed to be greater than (or equal to) ®% (50% is the exact valueproposed) of the corresponding average weight obtained by the LP model [formula-tion (2)]. For example, for the performance attribute y1, the constraint will be:

k1 ˆ ®

100

Xn

kˆ1

¬k;1

n;

where ¬k;1 is the weight of output 1 assigned by lathe k in the LP [formulation (2)]model. One can use this approach to reduce the level of subjectivity in constrainingweights.

After running Model I, we have obtained the results of e� ciency scores andweights for the 21 lathes in table 13 (where only output weights are listed). Thesedata represent the ratio between the output weight and price. The last row in table 13shows the average weight by factor, which is derived by averaging the weight valuesof each output.

2032 S. Sun

Lathe No. y1 weight y2 weight y3 weight y4 weight y5 weight y6 weight Score

1 0.000132 0 0.0108 0 0 0 12 0.0000286 0 0 0 0.00398 0 0.8383 0.000164 0 0 0.0171 0.00637 0 14 0.000168 0 0 0 0 0 15 0.0000556 0.0406 0 0.0489 0 0 0.9666 0 0.00762 0 0 0.00286 0 0.8197 0.000464 0.1062545 0 0 0.0275 0.0000738 18 0.000048 0.0278075 0.0157 0.00174 0 0 19 0.0000191 0 0.0239 0 0.000415 0.000721 1

10 0.000161 0 0 0 0.00960 0 0.86911 0 0.00714 0 0 0.00268 0 0.82112 0.0000135 0 0 0.0159 0.00217 0 113 0 0.00891 0.000810 0 0.00304 0 0.88914 0 0.0344 0 0.00966 0.000992 0 115 0 0.0576 0.0103 0 0 0 116 0 0.00788 0 0 0.00296 0 0.96617 0 0.0752 0 0 0.00282 0 0.92118 0 0.0533 0.00363 0.00775 0 0 0.89819 0.000123 0 0.0477 0 0 0.00184 120 0.000283 0 0 0 0.0169 0 121 0 0 0 0 0 0.000993 1

Weight 7.0133E-7 1.7097E-2 5.3733E-3 4.8119E-3 3.9184E-3 8.635E-5average byfactor

Table 13. Weights and pure technical e� ciency scores.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

In order to explore what happens to the outcomes, LP model (2)’s lower limitsare gradually modi®ed (in increments of 0.1), letting ® vary between 0 (the case ofstandard unbounded DEA) and 1 (the case where the lower bound is equal to theaverage of weights). Multiplying ® by the average weight by factor we obtain inferiorlimits for weights in `bounded DEA’. Table 14 shows the results of the trend ine� ciency as a function of r. This was repeated 11 times with the lower limits increas-ing progressively: at each iteration, e� ciency measurements for the whole sample oflathes were obtained from the application of LP model (2).

The weight restrictions imposed generally had no impact on the e� ciency ratingsof the lathes, as reported in table 14. Consequently, it is legitimate to wonderwhether excessive weight restrictions make it possible to draw valid conclusions inthis application. We acknowledge that this is not the general case, when constraintson weights usually a� ect e� ciency scores, sometimes dramatically.

We conclude that the weight restrictions using inferior limits had little e� ect onthe number of lathes that could be deemed to be 100% e� cient.

5.7. The ®nal selectionAccording to their higher cross-e� ciency mean scores, lower FPIs and maverick

index values, lathes 3, 4, 12, 14, 15 and 16 were identi®ed as possible systems forfurther consideration. Thus, we recommended these six lathes to the CSF for the®nal selection. Recall, in Phase IV, by law the CSF must use closed bidding to awarda contract to the lowest bidder who meets the desired speci®cations. Closed biddinginvolves a formal invitation to potential suppliers to submit written, sealed bids for aparticular business opportunity. All bids are opened and reviewed at the same time.

2033Assessing CNC machines using DEA

Multiplying factor used for computing weight inferior limits (®)Lathe

No. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1 1 1 1 1 1 1 1 1 1 1 12 0.838 0.838 0.838 0.838 0.838 0.838 0.838 0.838 0.838 0.838 0.8383 1 1 1 1 1 1 1 1 1 1 14 1 1 1 1 1 1 1 1 1 1 15 0.966 0.966 0.966 0.966 0.966 0.966 0.966 0.966 0.966 0.966 0.9666 0.819 0.819 0.819 0.819 0.819 0.819 0.819 0.819 0.819 0.819 0.8197 1 1 1 1 1 1 1 1 1 1 18 1 1 1 1 1 1 1 1 1 1 19 1 1 1 1 1 1 1 1 1 1 1

10 0.869 0.869 0.869 0.869 0.869 0.869 0.869 0.869 0.869 0.869 0.86911 0.821 0.821 0.821 0.821 0.821 0.821 0.821 0.821 0.821 0.821 0.82112 1 1 1 1 1 1 1 1 1 1 113 0.889 0.909 0.889 0.889 0.889 0.889 0.889 0.889 0.889 0.889 0.88914 1 1 1 1 1 1 1 1 1 1 115 1 1 1 1 1 1 1 1 1 1 116 0.966 0.966 0.966 0.966 0.966 0.966 0.966 0.966 0.966 0.966 0.96617 0.921 0.921 0.921 0.921 0.921 0.921 0.921 0.921 0.921 0.921 0.92118 0.898 0.898 0.898 0.898 0.898 0.898 0.898 0.898 0.898 0.898 0.89819 1 1 1 1 1 1 1 1 1 1 120 1 1 1 1 1 1 1 1 1 1 121 1 1 1 1 1 1 1 1 1 1 1

Table 14. Pure technical e� ciencies in function of weight inferior limits.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

Three members of the CSF buying centre evaluate each contract on several evalua-tion criteria. Although the number and nature of the criteria vary by product, ®ve

evaluation criteria are common: (1) the impact of the contract on plant capacity, (2)

product quality, (3) delivery requirements, (4) vendor reputation, and (5) sales sup-

port. The criteria are assigned weights based on their relative importance to the CSF.

Each of the contracts is rated on each of the previous criteria using a 0±10 scale, with

0 ˆ extremely poor, 5 ˆ average, and 10 ˆ outstanding. Summing the product of

each criterion weight and rating provides a total score. The contract is awarded tothe lowest bidder who has a vendor score between the ideal vendor score and the

minimum acceptable score.

In order to select the most suitable machine for acquisition, the CSF sent a

formal invitation to the manufactures of these six lathes. Table 15 represents the

results of the vendor assessment. The rating on each of the criteria for each vendor

2034 S. Sun

RatingLathe Evaluation criteria

Vendor No. (weight) High Medium Low Score

Yang Iron 3 Plant capacity (25) 6.7Product quality (20) 7.3Delivery (15) 6.3Vendor reputation (20) 7.6Sales support (20) 7.5 710

Victor 4 Plant capacity (25) 6.3Taichung Product quality (20) 6.5

Delivery (15) 7.7Vendor reputation (20) 7.6Sales support (20) 7.0 695

Tong Tai 12 Plant capacity (25) 4.6Product quality (20) 5.0Delivery (15) 5.6Vendor reputation (20) 5.3Sales support (20) 5.5 515

14 Plant capacity (25) 7.3Product quality (20) 7.6Delivery (15) 5.6Vendor reputation (20) 5.3Sales support (20) 5.5 706

15 Plant capacity (25) 7.6Product quality (20) 8.3Delivery (15) 5.6Vendor reputation (20) 5.3Sales support (20) 5.5 656

Matech 19 Plant capacity (25) 6.5Product quality (20) 6.7Delivery (15) 5.0Vendor reputation (20) 4.3Sales support (20) 4.5 525

Mean score 635

Note: Ideal vendor score is 1000; minimum acceptable score is 635.

Table 15. Evaluation of vendors.

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

was obtained by averaging the rating scores given by the three participants who tookpart in the ®nal selection process. It is clear from this table that lathes 3, 4, 14 and 15have total scores of 710, 695, 706 and 656, respectively; and these scores are betweenthe ideal vendor score and minimum vendor score. Among these four lathes, VictorTaichung, the manufacturer of lathe 4, provided the lowest bid and, thus, the opti-mal choice was VTURN 16. Finally, the contract was awarded to Victor Taichung.It is noteworthy that the ®nal choice was the same as the one previously identi®ed asthe best buy in the DEA analysis. We suggest that the DEA could be a good tech-nique for non-experts to use.

6. DiscussionThe proposed methodology can be applied from both a buyer’s and manufac-

turer’s perspective. The buyer (the CSF in this case) can use it as a justi®cation toolfor identifying `good-practice ’ machines in terms of technical speci®cations andcapital cost. With the review of ine� cient machines, it may be possible for thebuyer to negotiate changes in the cost-performance of these ine� cient machineswith their manufacturers that will make the machines competitive with e� cientmachines. However, the manufacturer s may not be willing to reduce the price of amachine.

The manufacturers can also use results from DEA in market competitive analy-sis. They may utilize the reference sets to identify competitors and niche players in acrowded market. The manufacturer who achieved an e� ciency score of 100, whencompared with other machines, can use these results to promoting its product. Onthe other hand, if a particular machine is poorly performing, then the manufacturerof this machine can use the analysis for benchmarking purposes (see Sarkis andTalluri 1996). We should note that the DEA only recommended one way to bee� cient. In practice, if a DMU really wants to be e� cient, it could take just onefactor that would be the easiest to improve and become e� cient, or it may improveonly those factors that the market views as important. DEA does not really tell usthis: this is a supplier or customer preferences issue.

The empirical study has shown that DEA can be seen as a normative model forhelping a buyer to evaluate advanced manufacturing machines. It does not requirean a priori weighting scheme to combine various dimensions of machine performanceinto an overall rating for each machine. However, the DEA model can be restruc-tured to allow for weight restrictions if necessary. DEA o� ers good support fortraining humans who must trust their judgements in the ®eld.

There are some limitations in the DEA model. First of all, DEA may be per-ceived as a `black box’ in terms of a method rationale if a decision-maker is unfa-miliar with linear programming concepts. Secondly, the amount of e� ort required toelicit information from a decision-maker is not large for DEA. Thirdly, like anyquantitative method, DEA relies heavily on what is included in the data set, i.e.garbage in, garbage out. Fourthly, there must also be rather more machines in theanalysis than measures or dimensions, otherwise the e� cient frontier will includemost of the machines. As Bowlin (1987) notes, the number of DMUs included in aDEA model should have almost twice the number of inputs and outputs. This makesit possible to reduce the number of degrees of freedom of DEA. Finally, DEA mayassign a set of DMUs with an e� ciency score of 1, which will reduce its discrimi-natory power. To remedy this problem, one may use the cross-e � ciency mean, FPI,

2035Assessing CNC machines using DEA

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

maverick index, and the bootstrap methods, as previously discussed in the precedingsections.

7. ConclusionsThis paper described a DEA application to the evaluation of CNC lathes in terms

of technical and cost factors. The application was in the context of a major studyinto AMT investments, undertaken by the CSF in 2000.

In this paper, we compared 21 CNC lathes by measuring the pure technicale� ciency for each, using the BCC model. We then estimated the cross-e� ciencymeans, FPI and maverick for each lathe in order to increase the discriminatorypower and make it possible to achieve a more `balanced view’ of lathe performance.We also carried out a sensitivity analysis with variable variation and weight restric-tions. Of 21 lathes, six were identi®ed as `good buys’ and recommended to the CSFfor further consideration for the ®nal selection of a vendor. Based on the results ofthe vendor assessment, the VTURN 16 was selected as the most suitable machine byconsidering plant capacity, product quality, delivery requirements, vendor reputa-tion, and sales support.

One of the limitations of this study is that the model does not capture all of thereasons for investing in CNC machines. However, we would point out that the modelcan easily take into account in the decision making process the case of e� ect ¯ex-ibility and strategic advantages. Another limitation is that the weight restrictions arenot derived from actual managerial experiences. We used average weights providedby the LP model as an alternative to subjective judgements on weights. Althoughweight restrictions may be needed, it is often di� cult to reach consensus on theprecise restrictions to use. A review of weight restrictions in DEA can be found inAllen et al. (1997).

Although previous DEA applications have con®rmed the applicability of DEAfor the evaluation and selection of industrial robots, cellular manufacturing systemsand FMSs, none of them has been applied to a real case. This paper illustrates itsimportance as a tactical/operational decision-making tool in the economic justi®ca-tion of CNC machines for the CSF using real data. The proposed approach can beused as an aid to strengthen managerial decision making in Phase III. In addition,potential uses of a DEA analysis of AMT systems might be: as a normative modelfor e� ectively discriminating between e� cient and ine� cient systems; in bargainingwith manufacturers; in competitor analysis, in determining unexplored marketniches; and as a basis for decisions about new product development.

As a ®nal comment, any analytic techniques cannot obviate the need for sub-jective judgement. In particular, we have to exercise judgement to limit the numberof inputs and outputs that enter the DEA analysis, otherwise nearly everyone willbecome 100% e� cient, and judgement is required to incorporate the most appro-priate variables in a DEA model (see Rouse et al. 1997). Judgement is also needed ininterpreting the results. Despite these quali®cations, we believe that the merits of amore quantitative approach, such as DEA, to compare machines will come to beappreciated by organizational buyers and such an approach will be used by them.

AcknowledgementsThe author would like to thank Mr J. E. Middle for editorial e� orts and the two

anonymous reviewers for their comments. Thanks also go to Dr Paul Rose whoassisted with the exposition of the paper. Any remaining errors are my own.

2036 S. Sun

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

References

AGRAWAL, V. P., KHOHL, V. and GUPTA, ?., 1991, Computer aided robot selection: a multipleattribute decision making approach. International Journal of Production Research, 29,1629±1644.

Albayrakoglu, M., 1996, justi®cation of new manufacturing technology: a strategicapproach using the analytical hierarchy process. Production and InventoryManagement Journal, 37(1), 71±77.

ALLEN, R., ATHANASSOPOULOS, A., DYSON, R. G. and THANASSOULIS, E., 1997, Weightsrestrictions and value judgments in data envelopment analysis: evolution, developmentand future directions. Annals of Operations Research, 73, 13±34.

BAKER, R. C. and TALLURI, S., 1997, A closer look at the use of data envelopment analysis fortechnology selection. Computers and Industrial Engineering, 32(1), 101±108.

BANKER, R. D. and THRALL, R. M., 1992, Estimating most productive scale size using dataenvelopment analysis. European Journal of Operational Research, 62, 74±84.

BANKER, R. D., CHARNES, A. and COOPER, W. W., 1984, Models for estimating technical andscale e� ciencies in DEA. European Journal of Operational Research, 30(9), 1078±1092.

BOWLIN, W. F., 1987, Evaluating the e� ciency of U.S. Air Force real-property maintenanceactivities. Journal of Operational Research Society, 38, 127±135.

BRAGLIA, M. and PETRONI, A., 1999, Evaluating and selecting investments in industrialrobots. International Journal of Production Research, 37(18), 4157±4178.

CHARNES, A., COOPER, W. W. and RHODES, E., 1978, Measuring the e� ciency of decisionmaking units. European Journal of Operations Research, 2, 429±444.

COOPER, W. W., SEIFORD, L. M. and TONE, K., 2000, Data Envelopment Analysis: AComprehensive Text with Models, Applications, References and DEA-Solver Software(Boston: Kluwer Academic).

DOYLE, J. and GREEN, R., 1991, Comparing products using data envelopment analysis.Omega, 19(6), 631±638.

DOYLE, J. and GREEN, R., 1994, E� ciency and cross-e� ciency in DEA: derivations, meanings,and uses. Journal of Operational Research Society, 45(5), 567±578.

DYSON, R.G. and THANASSOULIS, E., 1988, Reducing the weight ¯exibility in data envelop-ment analysis. Journal of Operational Research Society, 39(6), 567±578.

EFRON, B., 1979, Bootstrap methods: another look at the Jackknife, Annual Statistics, 7, 1±26.ELANGO, B. and MEINHART, W. A., 1994, Selecting a ¯exible manufacturing system ± a

strategic approach. Long Range Planning, 27(3), 118±126.FALKNER, C. H. and BENHAJLA, S., 1990, Multi-attribute decision models in the justi®cation

of CIM systems. Engineering Economist, 35(2), 91±114.KHOUJA, M., 1995, The use of data envelopment analysis for technology selection. Computers

and Industrial Engineering, 28(1), 123±132.KIM, G., PARK, C. S. and YOON, K. P., 1997, Identifying investment opportunities for

advanced manufacturing systems with comparative-integrated performance measure-ment. International Journal of Production Economics, 50, 23±33.

KLEINDORFER, P. R. and PARTOVI, F. Y., 1990, Integrating manufacturing strategy and tech-nology choice. European Journal of Operational Research, 47, 214±224.

KOLLI, S. and PARSAEI, H. R., 1992, Multi-criteria analysis in the evaluation of advancedmanufacturing technology using PROMETHEE. 14th Conference of Computers andIndustrial Engineering, Cocoa Beach, Florida.

KOLLI, S., WILHELM, M. R., PARSAEI, H. R. and LIES, D. H., 1992, A classi®cation schemefor traditional and non-traditional approaches to the economic justi®cation of advancedautomated manufacturing systems. In H. R. Parsaei, Economic and FinancialJusti®cation of Advanced Manufacturing Technologies (Amsterdam: Elsevier), pp. 165±187.

KUULA, M., 1993, A risk management model for FMS selection decisions: a multiple-criteriaapproach. Computer in Industry, 23(1/2), 99±108.

LEFLEY, F., 1996, Strategic methodologies of investment appraisal of AMT projects: a reviewand synthesis. Engineering Economist, 41, 345±363.

LOTFI, V. and SURESH, N. C., 1994, Flexible automation investments: a problem formulationand solution procedure. European Journal of Operational Research, 79, 383±404.

2037Assessing CNC machines using DEA

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

LUGGEN, W. W., 1994, Fundamentals of Computer Numerical Control, 3rd edn (Albany, NY:Delmar).

MEREDITH, J. and HILL, M. M., 1987, Justifying new manufacturing systems: a managerialapproach. Sloan Management Review, 28, 258±271.

MEREDITH, J. and SURESH, N. C., 1986, Justi®cation techniques for advanced manufacturingtechnologies. International Journal of Production Research, 24(5), 1043±1057.

MOHANTY, P. R. and VENKATARAMAN, S., 1993, Use of the analytic hierarchy process forselecting automated manufacturing systems. International Journal of Operations andProduction Management, 13(8), 45±47.

MYINT, S. and TABUCANON, M. T., 1994, A multiple-criteria approach to machine selectionfor ¯exible manufacturing systems. International Journal of Production Economics, 33,121±131.

NUNAMAKER, T. R., 1985, Using data envelopment analysis to measure the e� ciency of non-pro®t organizations: a critical evaluation. Managerial and Decision Economics, 6(1), 50±58.

PARKAN, C. and WU, M. L., 1999, Decision-making and performance measurement modelswith application to robot selection. Computer and Industrial Engineering, 36, 503±523.

PARSAEI, H. R., WILHELM, M. R. and KOLLI, S. S., 1993, Application of outranking methodsto economic and ®nancial justi®cation of CIM systems. Computer and IndustrialEngineering, 25(4), 357±360.

PROCTOR, M. D. and CANADA, J. R., 1992, Past and present methods of manufacturinginvestment evaluation: a review of the empirical and theoretical literature.Engineering Economist, 38(1), 45±58.

ROUSE, P., PUTTERILL, M. and RYAN, D., 1997, Towards a general managerial framework forperformance measurement: a comprehensive highway maintenance application. Journalof Productivity Analysis, 8(2), 127±149.

SARKIS, J., 1992, The evolution to strategic justi®cation of advanced manufacturing systems.In H. R. Parsaei, Economic and Financial Justi®cation of Advanced ManufacturingTechnologies (Amsterdam: Elsevier), pp. 141±163.

SARKIS, J., 1997, Evaluating ¯exible manufacturing systems alternatives using data envelop-ment analysis. The Engineering Economist, 43(1), 25±47.

SARKIS, J. and LILES, D., 1995, Using IDEF and QFD to develop an organizational decisionsupport methodology for the strategic justi®cation of computer integrated technologies.The International of Project Management, 13(3), 177±185.

SARKIS, J. and LIN, L., 1994, A general IDEF0 model for the strategic implement of CIMsystems. International Journal of Computer Integrated Manufacturing, 7(2), 100±115.

SARKIS, J. and TALLURI, S., 1996, E� ciency valuation and business process improvementthrough internal benchmarking. Engineering Valuation and Cost Analysis, 1, 43±54.

SARKIS, J. and TALLURI, S., 1999, A decision model for evaluation of ¯exible manufacturingsystems in the presence of both cardinal and ordinal factors. International Journal ofProduction Research, 37(13), 2927±2938.

SEXTON, T. R., 1986, The methodology of data envelopment analysis. In R. H. Silkman,Measuring E� ciency: an Assessment of Data Envelopment Analysis (San Francisco:Jossey-Bass), chapter 1, pp. 7±30.

SEXTON, T. R., SILKMAN, R. H. and HOGAN, A. J., 1986, Data envelopment analysis: critiqueand extensions. In R. H. Silkman, Measuring E� ciency: an Assessment of DataEnvelopment Analysis (San Francisco: Jossey-Bass), chapter 4, pp. 73±104.

SHAFER, S. and BRADFORD, J. W., 1995, E� ciency measurement of alternative machine com-ponent grouping solutions via data envelopment analysis. IEEE Transaction onEngineering Management, 42(2), 159±165.

SHANG, J. and SUEYOSHI, T., 1995, A uni®ed framework for the selection of a ¯exible manu-facturing system. European Journal of Operations Research, 85, 297±315.

SIMAR, L. and WILSON, P. W., 1998, Sensitivity analysis of e� ciency scores: how to bootstrapin nonparametric frontier models, Management Science, 44(1), 49±61.

SMITH, D. N. and EVANS, L., 1977, Management Standards for Computer and NumericalControl (Institute of Science and Technology, University of Michigan).

SON, Y. K., 1992, A comprehensive bibliography on justi®cation advanced manufacturingtechnologies. Engineering Economist, 38(1), 59±71.

2038 S. Sun

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014

STAM, A. and KUULA, M., 1991, Selecting a ¯exible manufacturing systems (FMS) usingmultiple criteria analysis. International Journal of Production Research, 29(5), 803±820.

STEFFY, S., SMITH, D. N. and SOUTER, D., 1973, Economic Guidelines for Justifying CapitalPurchases with Numerical Control Emphasis (University of Michigan).

SURESH, N. C., 1990, Towards an integrated evaluation of ¯exible automation investments.International Journal of Production Research, 28(9), 1657±1672.

SURESH, N. C., 1991, An extended multi-objective replacement model for automation invest-ments. International Journal of Production Research, 29(9), 1823±1844.

SURESH, N. C., 1992, A generalized multi-machine replacement model for ¯exible automationinvestments. IIE Transaction, 24(2), 131±143.

SURESH, N. C., 1994, Flexible automation investments: a problem formulation and solutionprocedure. European Journal of Operational Research, 79, 383±404.

SURESH, N. C. and KAPARTHI, S., 1992, Flexible automation investments: a synthesis of twomulti-objective modeling approaches. Computers and Industrial Engineering, 22(3), 257±272.

TABUCANON, M. T., BATANOV, D. N. and VERMA, D. K., 1994, A decision support system formulticriteria machine selection for ¯exible manufacturing systems. Computers inIndustry, 25(2), 131±143.

TALLURI, S. and YOON, K. P., 2000, A cone-ratio DEA approach for AMT justi®cation.Internal Journal of Production Economics, 66, 119±129.

TALLURI, S., HUQ, F. and PINNEY, W. E., 1997, Application of data envelopment analysis forcell performance evaluation and process. International Journal of Production Research,35(8), 2157±2170.

TALLURI, S., WHITESIDE, M. M. and SEIPEI, S., 2000, A nonparametric stochastic procedurefor FMS evaluation. European Journal of Operational Research, 124, 529±538.

TERBORGH, G., 1949, Business Investment Policy (Washington, DC: Machining and AlliedProducts Institute).

THYER, G. E., 1991, Computer Numerical Control of Machine Tools, 2nd edn (New York:Industrial Press).

2039Assessing CNC machines using DEA

Dow

nloa

ded

by [

Nor

thea

ster

n U

nive

rsity

] at

05:

55 1

3 O

ctob

er 2

014