Benchmarking Methodology for Railways Companies Trb2003-000966

8/10/2019 Benchmarking Methodology for Railways Companies Trb2003-000966

1/24

D.A. Tsamboulas, A. Frangos 1

BENCHMARKING METHODOLOGY FOR RAILWAYS COMPANIES

Paper no: 03-2966

Submission Date:31/ 07/ 2002 (initial), 14 November 2002 (in revised form)

Word Count: 7 500 words ( 5750 words text+ 1750 words for 7 tables)

Dr. D. Tsamboulas,Assoc. Professor

National Technical University of Athens, School of Civil Engineering, Department ofTransportation Planning and Engineering.5, Iroon Polytechniou Str., Zografou Campus, Zografou-Athens, GR-15773, GreeceTel.: +30-210-7721367, Fax: +30-210-7722404, E-mail: [email protected]

Andreas Frangos, Civil Engineer

12 Ipsiladou street, Kifissia, Athens 14561,Greecetel:+30-210-6201056, fax:+30-210-3815607,E-mail: [email protected]

Corresponding author:

Dr. D. Tsamboulas,Assoc. Professor

National Technical University of Athens, School of Civil Engineering, Department of

Transportation Planning and Engineering.5, Iroon Polytechniou Str., Zografou Campus, Zografou-Athens, GR-15773, GreeceTel.: +30-210-7721367, Fax: +30-210-7722404, E-mail: [email protected]

TRB 2003 Annual Meeting CD-ROM Paper revised from original submittal.


2/24


BENCHMARKING METHODOLOGY FOR RAILWAYS COMPANIES

ABSTRACT

The subject of this paper is the development of a methodology for benchmarking of railwayscompanies, with a focus on European railways. Due to the changing environment in Europe, thisis quite useful for Railway Managers, providing them with tools to assess the performance oftheir company in relation to others with similar characteristics and identify the areas forimprovement. The methodology develops modeling tools in order to perform a comparativeperformance assessment among railway companies. It consists of three stages. At the first stage,performance indicators are developed for assessment of companies' operations performance.Values of the performance indicators are produced for seventeen European railway companies.

At the second stage, with the use of appropriate indicators multiple DEA models are developed,which produce an overall efficiency score and identify the efficient and non-efficient railwaysfor the indicators considered. At the third stage, the appropriate changes are determined for eachnon-efficient company, with the use of the most representative model amongst the onesdeveloped. They are used to increase their efficiency and to identify the corresponding efficientcompanies to be used as benchmarked model-companies. Finally, with the methodologynumerical application valuable conclusions are drawn regarding the European railwaycompanies performance and methods to improve it.

Key words: Benchmarking, DEA, railway company, performance, efficiency.



3/24


1. INTRODUCTION

Benchmarking was formally recognised as a management tool in 1979, when Xeroxdeveloped it (1) and almost all successful US companies have been employing it since 1993 (2).There are numerous and different definitions of benchmarking, of which the most appropriate for

the purposes of the present paper is the following (3)

:

"benchmarking is the art of finding out, ina perfectly legal and aboveboard way, how others do something better than you do - so that youcan imitate - and perhaps improve upon - their techniques". Thus, benchmarking could provideinformation on how a particular high performance is achieved by other companies. Recently,benchmarking theory is shaping up for academic and practical purposes (4, 5). Benchmarkingtypes are grouped into seven categories (6):

Performance benchmarking

Process benchmarking

Strategic benchmarking

Internal benchmarking

Competitive benchmarking

Functional benchmarking

Generic (world class) benchmarking

From the above, the relevant to this paper is the performance benchmarking. Althoughempirical and scientific work is extensive, it has not been widely applied to railway companies.Hence most of the concepts used in this paper are drawn from relevant work in other sectors,such as by Kaplan (7) Kaplan and Norton (8), Pryor and Katz (9). Recent work for benchmarkingin the transport sector is done by Wouters et al. (10), and in the COMET study on metros (11).

The World Bank has published a study on re-engineering approaches in the railwaysector, with some examples on performance from key railways (12). In Europe, mostapplications for rail networks have been developed focusing on the technical aspects, and thusthey neglect the entrepreneurial aspect of rail transport. Furthermore, if any performanceassessment is done, the focus has been on internal performance and much less on comparisons

with other railways performance, except the usage of some indicators by the UIC (InternationalRailways Union), mainly for statistical purposes. This is due to the monopolistic character ofEuropean rail transport and the limitation of rail companies within their national borders. Theopening of the rail market in Europe and the limitation of the states participation have creatednew standards and have made the performance assessment of railway companies necessary. Arecent study for European Railways has been commissioned by UIC (13), which is moretechnically oriented, and a research project has been co-funded by the European Commission tolook among other issues- to the benchmarking of the European Railways (14). The introduction



4/24


of competition amongst Train Operating Companies (TOCs) in Europe, the assignment of theresponsibility for the infrastructure management and development, its maintenance andsignalling to Infrastructure Railway Companies (IRCs) and the need to attract financial fundsfrom private resources have made the improvement of their operations necessary.

The present paper aims at adapting the benchmarking process - which is applied to othersectors of the economy- to the railway sector with a focus to owners and managers ofinfrastructure, i.e. the European infrastructure railways companies. Hence, it develops abenchmarking methodology, which, with appropriate modeling tools, is able to produce conciseand useful results concerning the performance of European railway companies, and as such itwill assist the railways managers in improving the performance of their companies.

2. METHODOLOGY FRAMEWORK

The benchmarking methodology framework is designed mainly for infrastructurerailways companies (IRCs), since in a number of European countries separate companies handleinfrastructure and railway operations. Consequently, the IRCs are not competing directly withother transport modes, being a monopoly regarding national railway infrastructure management.However, train operating railways companies (TOCs) are in direct competition with othertransport modes. A prerequisite for the development of the methodology is some degree ofhomogeneity and similarity of the produced transport outputs and required resources-inputsamongst the companies considered. Hence, for the selection of railway companies' data, a test isperformed. However, the methodology is not able to handle concealed differences amongstrailways that might affect their performance. It is assumed that such differences are negligible,since all railways companies are within either the European Union (common laws, recentlycommon currency), the European Economic Area or the Accession countries of Eastern Europe(following the EU legislation in most cases).

The methodology consists of three stages. At the first stage, a performance indicatorsmodel is developed, which assess the performance of the operations of the companies, with theintroduction of the most suitable performance indicators (PI) for each relevant performancefactor. Values of the performance indicators are produced for seventeen European railwaycompanies. Table 1 presents the list of the railway companies, as well as the country where theyoperate and their abbreviated names. At the second stage, an appropriate combination ofperformance indicators is used in the development of multiple DEA models, which produce anoverall efficiency score for every railway company considered and identify the efficient and non-efficient railways. At the third stage, the in-depth development of a representative DEA model isdone, which is applied to each non-efficient company to determine: (i) the potential changes that

increase their efficiency and (ii) the corresponding efficient companies, being the benchmarkedmodel-companies. Finally, the numerical application of the models with real data producesvaluable conclusions about the European IRCs performance and methods to improve it.



5/24


3. DATA ENVELOPMENT ANALYSIS

The elaborated benchmarking methodology is greatly based on a non-parametricoperations technique, the Data Envelopment Analysis (15). It is a linear programming methodthat produces a single efficiency score concerning a decision-making unit (DMU), being the

railway company. It creates an efficiency frontier and the DMUs, which lie on it, are consideredto be efficient and with an efficiency score equal to one (16).The rest of the DMUs, which arelocated in the interior of the envelopment surface, are considered to be non-efficient and have anefficiency score between zero and one.

There are three basic orientations: input, output and combination of input/output. Eachone focuses on decreasing the inputs, on increasing the outputs or on a simultaneous change,respectively. Hence, the choice orientation depends on how easy is for the railway company tochange its inputs or outputs. To apply the DEA two approaches are widely used: CRS (constantreturns to scale) which is used if an increase in inputs leads to a proportionate increase in outputsand VRS approach (variable returns to scale) which is used if the produced increase in outputs is

not proportionate. Each approach leads to a different model. The elaborated methodology uses aninput-oriented model, as it is much easier for a railway company to change its resources than itsoutputs (e.g. transport volumes of passenger and freight) and it employs the CRS approach,which leads to more strict efficiency scores. The corresponding model is the following:

Minimize E

with respect to w1,w ,

Subject to:

K,1,k0)2(1

K=

=

N

j

knnkjj xExw

Where,N organizations in the sample producingIdifferent outputsyin denotes the observed amount of outputifor organizationnK different inputs

xkn denotes the observed amount of inputkfor organizationn).wj weights applied across theNorganizations.En* is the smallest numberEn

When the nth linear program is solved, these weights allow the most efficient method ofproducing organization ns outputs to be determined. The efficiency score for the nthorganization, En*, is the smallest number Enwhich satisfies the three sets of constraints listed

I,1,i0)1(1

K

==

inijj

N

J

yyw

N1,2,...,j0)3( =jw



6/24


above. For a full set of efficiency scores, this problem has to be solved Ntimes - once for eachorganization in the sample.

The methodology takes advantage of the DEA properties: handles multiple inputs oroutputs; does not require any functional form between inputs and outputs; includes variables

with different units; leads to a single efficiency indicator; reveals best practices and givesspecific suggestions for the increase of efficiency; DMUs are directly compared against a peer orcombination of peers; a pre-standardization of inputs and outputs is not needed. On the otherhand, the negative aspects are: (i) since DEA is an extreme point technique, a measurement errorin an indicator can cause significant problems; (ii) provides information on how well thecompany does compared to its peers but fails to do so regarding the "theoretical maximum"; (iii)does not classify the efficient DMUs; (iv) it is difficult to aggregate the results of the efficiencyestimations of a DMU. However, the application of DEA provides value added informationwhen compared with simple statistics: (i) it can incorporate multiple inputs and outputs andcalculate technical (or other type) efficiency; (ii) if regression analysis is used for the same data,DEA offers more accurate estimates of relative efficiency because of its boundary approach; (iii)

it is difficult with statistics to perform consistency tests with more than two parameters, wherecomparisons for efficiency can be obtained with DEA.

4. DEVELOPMENT OF PERFORMANCE INDICATORS

The first stage follows four logical steps: recognition of the IRCss strategy, determination of thecritical success factors, creation of performance indicators for each factor, selection of finalperformance indicators.

4.1 Strategy

The strategy followed by the European IRCs is the satisfaction of their direct clients(TOCs) and at the same time the balance of their accounts (as it is stipulated by EU legislation).However, this is not yet the practice in most railway companies, since they still belong to thestate and as such they must serve the social needs as well 1. Thus, the benchmarking methodologyfollows the above strategy environment for assessing the companies performance in theirattempt to balance their accounts.

4.2 Critical success factors

Success is referred to a set objective that usually falls within one of the following fourdimensions: internal performance, financial performance, customer satisfaction and learning andgrowth. Thus, for each of these dimensions success factor or factors are introduced, for assessingthe success. For the IRCs eight factors are identified which are critical for the success (17): Assetutilization; Financial performance; Security; Accessibility; Reliability and quality of services;Efficiency; Innovation & growth; Customer satisfaction.

1 The EU legislation provides for public servicecontracts for these cases in order to secure revenue for the railwaysregardless of the demand. However this is not widely applied.



7/24


4.3 Initial performance indicators

For the success factors, performance indicators are introduced. In developing theperformance indicators (PI) the first task is to decide what to compare. The methodologycompares European IRCs with the use of the success factors, and therefore the performance

indicators correspond to these factors. An initial set of such factors was derived from relevantresearch programs, such as Improverail (still in progress) (17), CoMET & NOVA (11), Equip(18), Europe-Trip (19) and Prorata (20). In this way, the initial PIs are created, which are usedfor further analysis.

4.4 Performance indicators

The benchmarking model developed is based on a limited number of performanceindicators, which assess a railways companys performance. They are selected from theidentified set of indicators, based on their compliance with the following three criteria:

1. Representativity: the ones assessing concisely and fully each success factor andexhibiting the less possible correlation with other selected PIs.2. Comparability: PIs not greatly affected by external factors and used for objective

comparison between companies.3. Accessibility of data: the ones requiring data publicly available or easily collected.

5. BENCHMARKING PERFORMANCE INDICATORS

The performance indicators -that meet the selection criteria- for each success factor arepresented hitherto. Some numerical examples are provided for the most important indicators, in

Tables 2 and 3.

5.1 Asset utilization

The companys goal is to use its assets as more intensively as possible, ensuring at thesame time the reliability, the security and the quality of service. The assets of an IRC in Europeconsist of the rail network, the stations, the signaling system but not the rolling stock. Howeverthe performance and satisfaction of the TOCs (direct clients) that own and operate the rollingstock affects the revenues of the IRC. The PIs considered are:

Length of network:expresses the network size, in line-kms, regardless of the track-kms.Train-km / km of lines: expresses the level of exploitation of the network by trains operated bythe TOCs.Passenger-km / km of lines: expresses the level of exploitation of the networks capacity bypassenger trains.Ton-km / km of lines: expresses the level of exploitation of the networks capacity by freighttrains.% of electrified lines:expresses the level of modernization of the rail network.% of double track lines:expresses the level of growth of the network lines.



8/24


5.2 Financial performance

The companys goal is to limit the operational costs and to increase its revenues with aparallel reduction in state subsidies. These three actions ensure the companys good financialperformance. The proposed PIs are:

Revenue / operational cost (includes maintenance of the tracks):expresses the ability to balancethe accounts and produce a profit.Revenue / km of lines: demonstrates the financial performance related to network size.Revenue / train-km: demonstrates the financial performance related to network transport volume.Investment cost / train-km: demonstrates the relation between the investment costs and theintensity of network usage.Operational cost / train-km: demonstrates how successful is the management regardingoperational costs according to the intensity of network usage.Operational cost / km of lines: demonstrates how successful is the management regardingoperational costs according to size of the network.

Liabilities / financial income: demonstrates how important are the liabilities in comparison to the

financial income.State subsidies / operational cost: shows how important for the company the state aid is, whichin Europe concerns the financing from the state for rail network extension, upgrading, but not formaintenance.

5.3 Safety

The companys objective is the decrease of accidents and in particular deaths andinjuries, which occur in the rail network. This includes the fatalities caused to the railwaypersonnel. The proposed PIs are:

Deaths / passenger-km: Presents the safety level (regarding deaths) for passengers and others(e.g road vehicles passengers, pedestrians) in relation to the passenger traffic on the network.

Injured / passenger-km: presents the safety level (regarding injuries) passengers and others inrelation to the passenger traffic on the network.

Accidents / train-km: shows the overall safety level of the network in relation to the rail traffic onthe network

Deaths of railways personnel / passenger-km: Presents the safety level (regarding deaths) for thepersonnel of railway company in relation to the passenger traffic on the network.

Injured railway personnel / passenger-km: presents the safety level (regarding injuries) for thepersonnel of railway company in relation to the passenger traffic on the network.

5.4 Efficiency

The companys goal is the offer the planned services with the minimum personnel andresource consumption. The proposed PIs are:



9/24


Lost working hours / total working hours: shows the consistency of the employees in respectingtheir official working hours.Personnel / km of lines: shows how efficient is the management regarding the persons employedaccording to the size of the network.Operating costs (including maintenance)/ personnel: shows how efficient is the management

regarding the persons employed, by presenting the average cost per employee

5.5 Accessibility

It measures the degree by which demands for slots of passenger and freight trains aresatisfied. The proposed PIs are:

Line kms/population: presents the accessibility of railway network to the populationLine kms/population density: presents the accessibility of railway network to the population witha consideration of the countrys territory surface

5.6 Reliability & Quality of service

It measures the degree by which high-quality services are provided to the TOCs. Theproposed PIs are:

Arrivals and departures on time / total number of departures and arrivals: shows how consistentare the timetables of departures and arrivals with the available capacity by the IRC.Train-hours of delay / total train-hours: shows the magnitude of the delays according to theoffered services

5.7 Innovation & growth

The development of new services and products to increase the number of clients andconsequently revenues. The proposed PIs are:

% of change of train-km from last year: demonstrates the growth of the company related to theoffered services.% of change of passenger-km from last year: demonstrates the growth of the company related tothe accommodated passenger demand% of change of ton-km from last year: demonstrates the growth of the company related to theaccommodated freight demand.% of income from new products or services: shows how much the company takes advantage ofnew products and services

5.8 Customer satisfaction

It measures the satisfaction of their clients needs, which are the TOCs (direct clients) andthe passengers or freight operators (indirect clients). The proposed PIs depend on the marketresearch to be performed.



10/24


5.9 Interpretation of PIs

The examination and interpretation of the PIs is an important part of the benchmarkingprocess. From the data of Tables 2 and 3 we can produce some valuable tentative conclusionsabout the performance of the European railway companies.

The exploitation of its network is very different and applies to a different level ofdevelopment of rail transport in each country. SBB is the champion in using intensively itsnetwork, being also the only company, which manages an almost entirely electrified network.The freight transport is more developed in eastern European companies. The financial conditionof most railway companies is not good as they have large debts, extensive operational costs andlimited revenues. The eastern European companies seem to keep a balance between theiroperational costs and their revenues and have limited liabilities in contrary to most E.U.companies, which can cover only a small percentage of their costs with their revenues. This ismainly due to heavy infrastructure investments that take place in the EU railway network. On theother hand, the subsidies provided by the state for large infrastructure construction like the Greek

high-speed network and the French TGV, result in a low subsidy/operating cost ratio for CH andSNCF, the latter if compared with DB. Concerning the safety of their networks, the EUcompanies have a relatively better performance and gather more systematically data than easternEuropean companies. RENFE has the best performance, though one-year data are not adequatefor a proper and fair estimation of the safety standards in its network. As for the efficiency, thereis the same variance between the companies. The Scandinavian companies (NSB, SJ, VR) appearas the most efficient mainly because of their personnels efficiency.

6. DEVELOPMENT OF MULTIPLE DEA MODELS

6.1 Basic principles

At the second stage, various DEA models with multiple variables and different prioritiesare developed. All models have some common features. They are input-oriented and assume aCRS approach. There are no constraints in the weights. The variables are concentrated on theresources/inputs and the results/outputs of the railway companies, but they include othercharacteristics. The development of DEA has been achieved with the help of the DEA-solvingprogram, which is called EMS (Efficiency Measurement System).

6.2 Infrastructure Railway companies (IRCs)

For models development a number of railways companies are considered. The aim is toinclude companies with common characteristics, and in particular to offer services for bothpassenger and freight transport that function in a common context. Hence, as a sample theseventeen railway companies from Europe are used (Table 1).

The criteria to test the assumption of common characteristics among the railwaysconsidered are the passenger-freight mix and the length of haul. The other characteristics are



11/24


common due to the nature of railways, being either from the European Union or EuropeanEconomic Area or East European accession countries This is proven by the following: thepassenger freight mix for the railways considered has a ratio of passenger-kms/ (passenger-kms+ ton-kms) 50% with the standard deviation of 19,7 (1998 data) , the ratio of passenger train-kms

/ (passenger train-kms + freight train kms) is 76,8 with a standard deviation of 9,7 (1998 data)

and it remained almost constant for the period 1993-2001 with an average 76,5 and a standarddeviation 8,8. Finally regarding the average length of haul for 1998 it was 56 kms for passengers(standard deviation 2,4), and for freight 228 kms (standard deviation 9).

6.3 Data sources and quantification of indicators

The reference year is 1998, since this is the most recent year that data exists for mostrailways. The sources of our data have been the official catalogues of UIC and when data werelacking other sources are used. Formal sanity tests for the UIC data were not carried out, butselective data for some railways were tested with the data collected for the purposes of the

IMPROVERAIL project (14) and the RAILISA project that collects data for the Europeanrailways. In addition, only the companies, which provided data to the UIC database consistentlyfor the past years were included in the models.

The quantification of indicators was done with the use of values for each of thecomponents of the indicator as they are retrieved from the data source. When ratios are needed,the relevant calculations were done before the value for the indicator is introduced in the model.Concerning the money values (reported in US $ in the UIC data) a correction was done with anindex to reflect the purchasing power parity of the monetary value amongst the differentcountries considered.

6.4 Purchasing Power Parity (PPP) use

The market exchange rates of the national currencies (average for 1998) in relation to theUS dollar are usually used to convert the different monetary units to a common currency.However, this does not reflect the real purchasing value of each amount. This shortcoming canbe fixed by using the PPP currencies. PPPs are the rates of currency conversion that equalize thepurchasing power of different currencies by eliminating the differences in price levels betweencountries (21). For this paper, the GDP PPPs are used, which are calculated as a geometricaverage of price relatives of various products that comprise the GDP of each country.

It is evident that the efficiency scores are affected significantly with the use of PPPs. Thecompanies, which function in countries with low price levels, get lower efficiency scores thanbefore. This is evident for the companies of Eastern Europe, which have a very low relativeposition in the sample. On the contrary, the companies, which function in countries where thereare high price levels maintain or improve their relative efficiency. In this way, the disparitiesbetween the different companies are reduced and the results are much more reliable.



12/24


6.5 Model development process

For models development all the suggested PIs are introduced initially, but based on themodels performance and validity of data for the values of the PIs, fewer are maintained. Theinputs (resources) of a railway company are: The operational Cost expressed as Operational

Cost/Km of lines and Operational Cost/personnel, the subsidies related to Operational Cost, thenumber of freight and passenger trains (in train Km) and the length of network in operation. Theresources are included as plain numbers in the model or as ratio in order to normalize the valuesamongst the railway companies. The use of ratios as variables is recommendable. The modelswith their use produce more logical and representative results. The introduction of plain numbersas variables leads to an increase in the number of the variables, which has as result a significantincrease of the efficiency scores and of the number of the efficient companies. In some cases theinputs need to be transformed to produce the right numerical dimension to all variables. This isdue either to their value or to what DEA models try to accomplish (max or min of the variable).As an example, the indicator used to express the length of network is the inverse, i.e. max (lengthof lines in operation), or equally min (1/length of lines in operation). The outputs (results) are:

Trains Km, Passenger Km and Tones Km all related to Km of lines and the accidents (fatalitiesof passenger and personnel) using the inverse indicators for the same reason as above. Theseindicators assess objectively the performance of the IRC.

Another comment is the classification of the companies as efficient and non-efficient,which is produced by the application of DEA. This should not be considered binding, but onlyindicative. A small change in the values or the selected variables may change significantly theefficiency scores.

7. PRESENTATION OF MODELS

Table 4 depicts the variables of every DEA model, whereas the produced efficiencyscores with the use of models are presented in Table 5. As it is stressed before, the efficiencyscores are very sensitive to the variables of the models and might affect the classification of thecompanies. Thus, the selection of the variables is very important for the success of the entireprocedure. It is also essential to define the denominator of the variables, since the inputs andoutputs change with the choice of the denominator, e.g. if the denominator is the size of thenetwork, the railway companies that use intensively their network appear efficient and if thedenominator is the trains capacity, the companies that manage productively their rolling stockget the high efficiency scores. A brief description of each developed model follows.

Model 1

It demonstrates the efficiency of railways companies using as inputs economic factors withindicators: Operational Cost/Km of lines, Operational Cost/ Personnel, Subsidies/ OperationalCost. As outputs they have trains Km/ km of lines, which indicates their production. Sixcompanies characterized as efficient, whereas the rest have a score of 46% to 96 %. This modelis more appropriate for Infrastructure Railway Companies.



13/24


Model 2

It employs the same indicator as output as Model 1, but the inputs are asset factors withindicators passenger, freight trains and network lines in operation. The results vary from model1. Only one company (the German DB AG) is characterized efficient, whereas the rest have ascore of 2% to 58 %. The model is more appropriate for Train Operating Companies.

Model 3

It includes as inputs, economic factors with indicators Operational Cost/ Km of lines, Subsidies /Operational Cost and as outputs the asset factors with indicators Passenger and Freight trains,using the same output as the previous models, trains Km / km of lines. Five companies are foundefficient, whereas for the non-efficient ones the score varies from 41% to 96%. This model ismore appropriate when a railway company performs both tasks, those of train operator andinfrastructure manager.

Model 4

It is an extension of model 1, whereas the length of network in operation is added in inputs. The

results are similar to model 1. Companies with large network sizes have improved efficiency inthis model.

Model 5

It contains the resource indicators used in model 2 and as outputs uses two factors; PassengerKm/ km of lines and Tones Km/ Km of lines. They are chosen in order to assess for everycompany, which sector (passenger or freight) need improvements. The results show two efficientcompanies and the rest with scores 2% to 86%.

Model 6

It has the same inputs of model 3 and outputs of model 5. Seven companies are found efficient,whereas the non-efficient ones have scores from 45% to 91%. The results as expected are quitesimilar to model 3.

Model 7

It uses as resources those of model 1 and as outputs the ones of model 5. The results indicatenine efficient companies and non-efficient scores from 50 to 82%.

Model 8

It uses as resources those of model 1 and as output the accident indicators (fatalities of passengerand fatalities of staff). The results show four efficient companies and scores for non-efficientones from 2% to 72%.

Model 9

It contains the inputs of model 2 and as outputs the accident indicators. The results show fourefficient companies and scores for non-efficient ones from 23% to 86%.



14/24


Model 10

It contains the inputs of model 2 and uses as outputs trains-Km / km of lines and an accidentfactor (fatalities of employees). The results show two efficient companies and scores for non-efficient companies from 13% to 87 %.

8. PRODUCTIVITY RELATED MODEL

At stage three, DEA models are used to provide more in depth analysis than a simpleefficiency score. Hence, a representative model of the ones produced, model 4, was chosen forfurther analysis. This is justified, since it places emphasis on Railway Infrastructure Companiesand includes the main economic factors and the length of network in operation (as resources) andas outputs the productivity factor (trains km / km of lines).

Based on the developed model some suggestions are possible, presented in Table 4, sothat the non-efficient companies could become more efficient. The proposed changes are

indicative and are based solely on the mathematical model results. Table 6 shows for eachrailway company the extra amounts by which the inputs can be reduced or the outputs beincreased to achieve optimal results. In addition, they reveal the most severe problems, such asthe high level of subsidies, the length of network in operation and the Operational Cost/ Km oflines, which may assist the decision-making procedure in a specific company.

Also, the DEA application defines model-efficient companies for each non-efficientcompany. Practically, it distinguishes the efficient companies, which have the most commoncharacteristics with every non-efficient company and suggests them as benchmarks for furthercomparison. Table 7 shows what are the model-companies and their relative importance in theso-called peer group as it is derived from the weights in the parenthesis. As for the efficientcompanies, it is observed how many times each of them is a benchmark for the non-efficientones.

9. CONCLUSIONS

The proposed methodology, that includes the PIs and DEA models, provides A completebenchmarking framework, able to produce an indicative ranking of the railway companies, todetect problems and to suggest possible solutions. The benchmarking indicators help tounderstand the different dimensions of the companys operations and are used as variables forthe DEA models. The application of DEA models concentrates on different aspects of efficiencyand provides the tool to select the most representative model(s), which employs the mostessential variables. Finally, the numerical application of PIs analyzes the performance of all thecompanies important operations and assist in identifying areas for potential improvement. It isworth mentioning that the methodology is applicable to the railway companies, without anyserious discrimination between IRCs and TOCs, and its principles could be applicable to anytransport company.



15/24


However, it should be recognized that the developed DEA models are only the initialphase of a benchmarking procedure. Data collection and elaboration is an important task, and assuch it should be the basis for the search of best practices and solutions that may be applicable toa specific company to increase its performance. The liberalization of rail transport and thesubsequent incorporation of efficiency optimization principles require continuous performance

evaluation. The incorporation of the benchmarking procedure in the common entrepreneurialprocedures of the railway companies is a first step that may give to the procedure the appropriateattention from the management, which can eventually assist in the development of the entirerailway transport sector.

REFERENCES

1. Malcolm Baldridge National Quality Award Application Abstract., Xerox cooperation,Rochester NY, 1989.

2. Karlf Bengt and stblom Svante , Benchmarking: A Signpost to Excellence in Quality andProductivity. John Wiley & Sons, Chichester, 1993.

3 Main John, How to steal the best ideas around.. Fortune, 126, (8), 1992.

4. Andersen B. and Pattersen P., Benchmarkin Handbook. Kluwer, 1995

5. European Commission, Thematic Network VERITE, International Conference onBenchmarking, Stuttgart, June 2002.

6. Bogan, C.E., English, M.J. Benchmarking for best practices. McGraw Hill, 1994.

7. Kaplan, R.S., (ed.), Measures for manufacturing excellence. Boston, Massachusetts, HarvardBusiness School Press, 1990

8. Kaplan, R.S. and Norton, D. P., Putting the balanced scoreboard to work. Harvard BusinessReview, September-October 1993, pp134-147.

9. Pryor, L.S. and Katz S.J., How benchmarking goes wrong (and how to do it right). PlanningReview, 21(1), 1993, pp. 6-11

10. Wouters Marc, Kokke Kees, Theeuwes Jacques and Van Donselaar Karel, Identification ofcritical operational performance measures a research note on a benchmarking study in the

transportation and distribution sector. Management Accounting Research, 10, 1999, pp439-452

11. Powers V. J., Benchmarking in Hong Kong: Mass transit Railway excels in worldwideindustry study. Benchmarking in Practice, 11, American Productivity and Quality Center,1998



16/24


12. Kopicki R. and Thomson L., Best Methods of Railway Restructuring and Privatization, CFSDiscussion Paper Series, No.111, World Bank, Washington D.C., 1997

13. UIC- Booz Allen Hamilton, European Railways Performance Regime Study, Paris 2001

14. European Commission, IMPROVERAIL Research Project of the 5

th

FrameworkProgramme on Research, Brussels, 2000

15. Charnes, A., Cooper, W.W., Rhodes, E. Measuring the efficiency of decision-making units.European Journal of Operational Research 2, 429-444, 1978.

16. Martin, J.C., Roman, C., An application of DEA to measure the efficiency of Spanishairports prior to privatization. Journal of Air Transport Management 7, 149-157, 2001.

17. European Commission, IMPROVERAIL, Research Project. Deliverable 2, BenchmarkingMethodologies and Harmonization of Concepts in the Railway Sector, Brussels, 2001.

18. European Commission, EQUIP Research Project, Extending the Quality of Public Transport.University of Newcastle, 2000.

19. EUROPE-TRIP, European Railways Optimization Planning EnvironmentTransportationRailways Integrated Planning. Ferrovie dello Stato SpA, Rome, 2000.

20. European Commission, PRORATA Research Project, Profitability of Rail Transport andAdaptability of Railways.Brussels, 2000.

21. Schreyer, P., Koechlin, F., Purchasing power parities-measurement and uses. Statistics Brief3, 1-8, 2002.



17/24


List of Tables

TABLE 1 - Railways Companies used in DEA modelsTABLE 2 - Asset utilization and financial PIs

TABLE 3 - Safety and efficiency PIsTABLE 4 - Variables of DEA modelsTABLE 5 Ranking and efficiency scores of DEA modelsTABLE 6 - Quantitative suggestions for the increase of efficiencyTABLE 7 - Model companies



18/24


Table 1 Railways Companies used in DEA models

Company Country Abbreviation

1 Organisme de Chemins de fer helleniques Greece CH

2 Coras Iompair Eireann / Irish TransportCompany

Ireland CIE

3 Caminhos de Ferro Portugueses Portugal CP

4 Deutsche Bahn AG Germany DB AG

5 Ferrovie dello Stato Italy FS

6 Osterreichische Bundesbahnen Austria BB7 Red Nacional de los Ferrocarriles

EspanolesSpain RENFE

8 Statens Jarnvagar Sweden SJ

9 Societe Nationale des Chemins de ferBelges

Belgium SNCB/NMBS

10 Societe Nationale des Chemins de fer

Francais

France SNCF

11 VR Yhtyma Oy Finland VR

12 Chemins de fer federaux suisses Switzerland CFF/SBB/FFS

13 Norges Statsbaner BA Norway NSB BA

14 Ceske drahy Czech Republic CD

15 Magyar Allamvasutak Hungary MAV

16 Polskie Koleje Panstwowe Poland PKP

17 Eisenbahnen der Slowakischen Republik Slovak Republic ZSR



19/24


TABLE 2 Asset utilization and financial PIs

ASSET UTILIZATION PI FINANCIA

Companies

Train-km / kmof lines(*1000)

Passenger-km / km of

lines(*1000000)

Ton-km/kmof lines

(*1000000)

% ofelectrifi

edlines

% ofdouble

track

Revenue/Operational

cost

Revenue /Train km

(*1000)

Operationalcost/train km

(*1000)

Oc

(

CH 7,211 0,675 0,142 0,000 13,963 0,003 0,000 0,032

CIE 8,102 0,744 0,244 1,938 25,668 1,427 0,031 0,022

CP 19,175 1,647 0,733 31,246 0,006 0,000 0,069

DB AG 22,870 1,552 1,931 49,460 45,714 0,598 0,011 0,018

FS 19,920 2,579 1,396 65,224 38,153 0,001 0,000 0,035

OBB 26,197 1,491 2,710 62,544 33,059 0,081 0,002 0,025 RENFE 14,067 1,481 0,950 54,647 28,849 0,007 0,000 0,019

SJ 9,906 0,695 1,416 74,347 1,000 0,012 0,012

SNCB/NMB

S

27,342 2,081 2,229 73,636 77,038 0,028 0,001 0,030

SNCF 15,834 2,023 1,700 44,547 0,170 0,005 0,028

VR 7,582 0,576 1,685 37,447 0,187 0,003 0,015

CFF/SBB/F

FS

40,447 4,290 3,003 99,725 1,000 0,025 0,025

NSB BA 9,340 0,647 0,604 61,308 1,000 0,014 0,014 CD 14,488 0,742 1,940 31,644 20,573 1,000 0,021 0,021

MAV 11,119 0,857 0,890 32,231 16,630 1,000 0,025 0,025

PKP 11,743 0,886 2,625 50,039 38,229 1,000 0,023 0,023

ZSR 16,136 0,850 3,206 41,860 27,788 1,000 0,031 0,031

AVERAGE 16,558 1,401 1,612 47,755 33,242 0,559 0,012 0,026 ST. DEV 8,7949 0,958 0,941 25,180 17,427 0,509 0,0117 0,013

Note: All monetary units in $

TRB 2003 Annual Meeting CD-ROM Paper revised from orig


20/24

D.A. Tsamboulas, A. Frangos

TABLE 3 Safety and efficiency PIs

SAFETY PI EFFICIE

Companies

Passenger

deaths /Passenger-

km(*1/1000000)

Injured

passengers /Passenger-

km(*1/1000000)

Accidents /Train-km(*1/1000)

Personnel deaths/Train-km

(*1/1000000000)

Wages /Personnel

Personnel /

Train-km(*1/1000)

CH 0,011598 0,089562 0,0330 0,656

CIE 0,002111 0,014778 0,064658 0,0170 0,690

CP 0,0240 0,198

DB AG 0,001926 0,002044 0,010611 0,020644 0,0490 0,240

FS 0,000386 0,001471 0,004991 0,006244 0,0500 0,367

OBB 0,000502 0,002760 0,014553 0,014283 0,382

RENFE 0,000057 0,000057 0,0390 0,214

SJ 0,112

SNCB/NMBS 0,000423 0,002254 0,010709 0,0530 0,429

SNCF 0,000218 0,000405 0,003568 0,003980 0,0510 0,349

VR 0,002961 0,002665 0,011253 0,022482 0,0320 0,314

CFF/SBB/FFS 0,000481 0,000561 0,003444 0,016992 0,262

NSB BA 0,0490 0,179

CD 0,003857 0,047136 0,0080 0,669

MAV 0,002253 0,033188 0,069830 0,023152 0,0080 0,663

PKP 0,000146 0,024571 0,007338 0,0090 0,799

ZSR 0,897946 0,016901 0,0060 0,835

AVERAGE 0,001023 0,005255 0,085929 0,019667 0,0305 0,432

ST. DEV 0,001060 0,00929593 0,235256 0,0172151 0,01833 0,235

Note: All monetary units in $



21/24


TABLE 4 Variables of DEA models

M1 M2 M3 M4 M5 M6 M7 M8 M9 M10

INPUTS

Operational Cost/ Km of lines x x x x x x

Operational cost/ personnel x x x x

Subsidies / operational cost x x x x x x

Passenger trains Km x x x x x x

Freight trains Km x x x x x x

Length of network in operation x x x x x x

OUTPUTS

Trains Km / Km of lines x x x x x

Passenger Km / Km of lines x x x

Tones Km / Km of lines x x x

1/ accidents (fatalities passengers +others) x x

1/ accidents (fatalities of railway employees) x x x



22/24


TABLE 5 Ranking and efficiency scores of Railways Companies in DEA m

M1 M2 M3 M4 M5 M6 M7 Rail Company

a b a b a b a b a b a b a b

CH 16 0,47 16 0,02 16 0,41 16 0,47 16 0,02 16 0,45 16 0,50

CIE 13 0,78 17 0,02 13 0,63 14 0,78 17 0,02 11 0,71 11 0,78

CP 1 1,00 13 0,06 1 1,00 1 1,00 14 0,08 1 1,00 2 1,00

DB AG 8 0,94 1 1,00 2 1,00 4 1,00 1 1,00 6 1,00 6 1,00

FS 15 0,57 3 0,37 15 0,55 15 0,64 4 0,69 9 0,86 7 1,00

OBB 10 0,87 5 0,29 11 0,71 11 0,90 6 0,35 14 0,58 15 0,62

RENFE 14 0,70 7 0,19 9 0,91 10 0,90 7 0,27 3 1,00 8 1,00

SJ 10 0,11 10 0,19

SNCB/NMBS 11 0,86 9 0,13 14 0,60 12 0,86 11 0,15 15 0,56 13 0,71

SNCF 9 0,88 2 0,58 4 1,00 7 1,00 2 1,00 2 1,00 4 1,00VR 5 1,00 14 0,05 3 1,00 8 1,00 12 0,13 5 1,00 5 1,00

CFF/SBB/FFS 6 1,00 6 0,25 8 0,93 5 1,00 5 0,38 8 0,91 1 1,00

NSB BA 7 0,96 15 0,04 6 0,96 9 0,96 15 0,04 10 0,82 10 0,82

CD 2 1,00 8 0,16 10 0,82 2 1,00 8 0,25 12 0,69 12 0,74

MAV 12 0,79 11 0,10 12 0,65 13 0,79 13 0,11 13 0,61 14 0,70

PKP 3 1,00 4 0,31 7 0,96 3 1,00 3 0,83 4 1,00 3 1,00

ZSR 4 1,00 12 0,08 5 1,00 6 1,00 9 0,19 7 1,00 9 1,00

Note: (a) Ranking Scores; (b) Efficiency scores. With bold the numbers that characterize efficient c



23/24


TABLE 6 Quantitative suggestions for the increase of efficiency

No Railway

Company Score

OperationalCost/ Personnel

length of networkin operation

Operational Cost/Km of lines

SubsidOperatio

1 CH 46,79% 0 0,087741 0 02 CIE 77,51% 0 0,300504 0

3 CP 100,00%

4 DB AG 100,00%

5 FS 63,69% 0 0 0,045845 0

6 OBB 89,65% 0 0 0,023341 0

7 RENFE 90,15% 0,032277 0 0

8 SNCB/NMBS 86,39% 0 0 0,073073

9 SNCF 100,00%

10 VR 100,00%

11 CFF/SBB/FFS 100,00%

12 NSB BA 96,18% 0,040592 0,065743 0 0

13 CD 100,00%

14 MAV 78,75% 0 0 0

15 PKP 100,00%

16 ZSR 100,00%



24/24


TABLE 7 Model-companies

No Railway

Company

Score Benchmarks-model companies

1 CH 46,79% 3 (0,28955) 13 (0,11446)

2 CIE 77,51% 3 (0,25375) 13 (0,05393) 15 (0,20902)

3 CP 100,00% 5

4 DB AG 100,00% 3

5 FS 63,69% 4 (0,58044) 15 (0,56592)

6 OBB 89,65% 13 (1,35409) 15 (0,56025)

7 RENFE 90,15% 3 (0,17934) 4 (0,46473)

8 SNCB/NMBS 86,39% 11 (0,02234) 13 (1,44909) 16 (0,33736)

9 SNCF 100,00% 0

10 VR 100,00% 0

11 CFF/SBB/FFS 100,00% 1

12 NSB BA 96,18% 3 (0,48710)13 CD 100,00% 5

14 MAV 78,75% 3 (0,16293) 4 (0,00692) 13 (0,26687) 15(0,33810)

15 PKP 100,00% 4

16 ZSR 100,00% 1

Documents

Benchmarking Methodology for Railways Companies Trb2003-000966