1100215X-main

7/28/2019 1100215X-main

http://slidepdf.com/reader/full/1100215x-main 1/12

Scientia Iranica E (2011) 18 (6), 1579–1590

Sharif University of Technology

Scientia IranicaTransactions E: Industrial Engineering

www.sciencedirect.com

A tool to evaluate the business intelligence of enterprise systems

M. Ghazanfari, M. Jafari, S. Rouhani∗

Department of Industrial Engineering, Iran University of Science and Technology, Tehran, Iran

Received 28 February 2011; revised 31 July 2011; accepted 10 September 2011

KEYWORDS

Business intelligence;Decision support;Enterprise systems;Evaluation tool.

Abstract Most organizations still experience a lack of Business Intelligence (BI) in their decision-making processes when implementing enterprise systems, such as Enterprise Resource Planning (ERP),Customer Relationship Management (CRM), and Supply Chain Management (SCM). Consequently, amodel and techniques to evaluate and assess the intelligence-level of enterprise systems can improvedecision support. This paper proposes an expert tool to evaluate the BI competencies of enterprisesystems, and combines a comprehensive review of recent literature with statistical methods for factoranalysis. A statistical analysis has identified six factors for the evaluation model: ‘‘Analytical andIntelligent Decision-support’’, ‘‘Providing Related Experimentation and Integration with EnvironmentalInformation’’, ‘‘Optimization and Recommended Model’’, ‘‘Reasoning’’, ‘‘Enhanced Decision-makingTools’’, and finally, ‘‘Stakeholder Satisfaction’’. Utilizing the extracted loads of each unique criterion, theintelligence of thework systems canbe measured anddepictedon sixdashboards, based on correspondingfactors, actualizingan experttool that can diagnose the intelligencelevel of enterprisesystems. Enterprisescan use this approach to evaluate, select, and buy software and systems that provide better decisionsupport for their organizational environment, enabling them to achieve competitive advantage.

© 2012 Sharif University of Technology. Production and hosting by Elsevier B.V. All rights reserved.

1. Introduction

Nowadays, informationand knowledge represent the funda-mental wealth of an organization. Enterprises try to utilize thiswealth to gain competitive advantage when making importantdecisions. Enterprise software and systems include EnterpriseResource Planning (ERP), Customer Relationship Management(CRM), and Supply Chain Management (SCM) systems. Thesesystems convert and store the data in their databases; there-fore, they can be used as a pool of data to support decisions andexplore applicable knowledge [1,2]. With the potential to gain

∗ Correspondence to: Department of Industrial Engineering, Iran Universityof Science and Technology, No. 43, Reyhani Pamchi Allay, Allameh Amini St,West Mobarez St,Abouzar Blv17789-14361,Pirouzi Ave, Tehran, Iran. Tel.: +989122034980; fax: +98 2177959502.

E-mail address: [email protected] (S. Rouhani).

competitive advantage when making important decisions, it isvital to integrate decision support intothe environment of theirenterprise and work systems. Business Intelligence (BI) can beembedded in these enterprise systems to obtain this competi-tive advantage [3,4].

In the past, Decision-Support Systems (DSS) were indepen-dent systems within an organization and had a weak rela-tionship with other systems (island systems). Now, enterprisesystemsare the foundation of an organization, andpractitionersdesign and may implement BI as an umbrella concept to cre-ate a comprehensive decision-support environment for man-agement [1,5]. Based on the ideas of Alter [1], and the researchcarried out on the non-functional requirements of enterprisesoftware and systems by Jadhav and Sonar [6,7] and also bySen et al. [8], today’s approach to decision support as a sepa-rate,individualsystem, such asDSS,has been replaced by a newapproach. This new approach creates an integrated decision-support environment, and takes the intelligence requirementsof enterprise systems into consideration. Kahraman et al. [9]have also discussed the roles of intelligence techniques to ob-tain a successful business strategy in enterprise informationsystems.

The evaluation of enterprise software and business systemsrequires models and approaches that consider intelligencecriteria,as well as the enterprise traditional functional and non-functional requirements and criteria. There have been somelimited efforts to evaluate BI, but they have always considered

1026-3098 © 2012 Sharif University of Technology. Production and hosting byElsevier B.V. All rights reserved. Peer review under responsibility of Sharif University of Technology.

doi:10.1016/j.scient.2011.11.011

http://dx.doi.org/10.1016/j.scient.2011.11.011

http://www.sciencedirect.com/

mailto:[email protected]



mailto:[email protected]

http://www.sciencedirect.com/


7/28/2019 1100215X-main


1580 M. Ghazanfari et al. / Scientia Iranica, Transactions E: Industrial Engineering 18 (2011) 1579–1590

BI a system that is isolated from other enterprise systems.Taking a global view, Lönnqvist and Pirttimäki [5] designed BIperformance measures, but before their effort, measurementand evaluation in the BI field were restricted to proving theworth and value of BI investment. Elbashir et al. [10] discussedmeasuring the effects of BI systems on the business process,and presented effective methods of measurement. Lin et al. [11]

have also developed a performance evaluation model for BIsystems using ANP, but they have also treated BI as a separatesystem.

A recent research review [6], which reports a systematicreview of published papers about evaluating and selectingsoftware packages and enterprise systems, concludes that thereis no comprehensive list of criteria for this evaluation. Pastresearch has paid little attention to intelligence criteria andhas not created models to evaluate these criteria. Our currentresearch addresses these needs in the field of evaluation of theintelligence of enterprise software and systems.

Organizations usually assess their functional and non-functional requirements in order to evaluate and select enter-prise systems, so consideration of their decision-support needsas a non-functional requirement raises the following questions:

RQ1: What are the evaluation criteria for the BI competency of enterprise systems and software?

RQ2: What is the fundamental structure of these criteria?RQ3: How can organizations evaluate and select their en-

terprise systems and software according to intelligencecriteria?

This research was carried out to find answers to the abovequestions and to provide an approach and a tool for efficientdecision support by evaluating the intelligence of businesssystems. The rest of this paper is organized as follows: Section 2consists of a literature review of BI definitions and evaluationsfrom managerial and technical approaches, and as an enablerof enterprise systems. A wide-ranging literature review about

BI and decision-support criteria to evaluate enterprise systemsis also summarized in Section 2. Section 3 discusses the researchobjectives and methodology. Section 4 describes the empiricalresults and an analysis including hypothesis testing of the mainresearch question, of factor extraction and labelling. A practical‘‘BI Evaluation Tool’’ was developed to evaluate the intelligencelevel of enterprise systems, based on the six correspondingfactors, and this tool is demonstrated in Section 5. Finally,Section 6 concludes the research work and its main results andlimitations, and proposes directions for future research.

2. Literature review

Business Intelligence or BI is a grand, umbrella term,

introduced by Howard Dresner of the Gartner Group, in 1989,to describe a set of concepts and methods to improve businessdecision making by using fact-based, computerized supportsystems [12]. The first scientific definition by Ghoshal andKim [13] referred to BI as a management philosophy andtool that helps organizations to manage and refine businessinformation for the purpose of making effective decisions.

BI was considered to be an instrument of analysis, providingautomated decision making about business conditions, sales,customer demand, product preference and so on. It useshuge-database (data-warehouse) analysis, and mathematical,statistical and artificial intelligence, as well as data miningand On-Line Analysis Processing (OLAP) [14]. Eckerson [15]understood that BI must be able to provide the following tools:

production reporting, end-user query and reporting, OLAP,dashboard/screen tools, data mining tools, and planning andmodelling tools.

BI includes a set of concepts, methods and processes toimprove business decisions, using information from multiplesources and applying past experience to develop an exactunderstanding of business dynamics [16]. It integrates the

analysis of data with decision-analysis tools to provide the rightinformation to the right persons throughout the organization,with the purpose of improving strategic and tactical decisions.A BI system is a data-driven DSS that primarily supports thequerying of a historical database and the production of periodicsummary reports [2].

Lönnqvist and Pirttimäki [5] stated that the term, BI, can beused when referring to the following concepts:

1. Related information and knowledge of an organization,which describe the business environment, the organizationitself, the conditions of the market, customers and competi-tors and economic issues;

2. Systemic and systematic processes by which organizationsobtain, analyse and distribute the information for making

decisions about business operations.A literature review around the theme of BI shows ‘‘division’’between technical and managerial viewpoints, tracing twobroad patterns. The managerial approach sees BI as a processin which data gathered from inside and outside the enterprise,are integrated in order to generate information relevantto the decision-making process. Here, the role of BI is tocreate an informational environment in which operational datagathered from Transactional Processing Systems (TPS) andexternal sources can be analysed in order to extract ‘‘strategic’’business knowledge to support the unstructured decisions of management.

The technical approach considers BI as a set of tools thatsupports the process described above. The focus is not on theprocess itself, but on the technologies, algorithmsand tools thatenable the saving, recovery, manipulation and analysis of dataand information [17].

However, in the overall view, there aretwo importantissues.First, the core of BI is the gathering, analysis and distributionof information. Second, the objective of BI is to support thestrategic decision-making process.

By strategic decisions, we mean decisions related to im-plementation and evaluation of organizational vision, mission,goals and objectives with medium to long-term impact on theorganization, as opposed to operational decisions, which areday-to-day in nature and more related to execution [17].

Bose [18] also describes the managerial view of BI as aprocess to get the right information to the right people at the

right time, so they can make decisions that ultimately improvethe performance of the enterprise.

The technical view of BI usually centres on the processesor applications and technologies for gathering, storing andanalysing data, and forproviding accessto datato help manage-ment make better business decisions. Another important ob-servation in BI evolution is that industry leaders are currentlytransitioning from operational BI of the past to analytical BI of the future, which focuses on customers, resources and capa-bilities, to influence new decisions on an everyday basis. Theyhave implemented one or more forms of advanced analyticsfor meeting these business needs. Ranjan [19] considers BI asthe conscious methodical transformation of data from any andall data sources into new forms to provide information that is

7/28/2019 1100215X-main


M. Ghazanfari et al. / Scientia Iranica, Transactions E: Industrial Engineering 18 (2011) 1579–1590 1581

Table 1: BI definitions.

BI definition Managerial approach Technical approach System-enabler approach

Focus Excellence of managementdecision-making process

Tools that support the process of BImanagerial Approach

Value-added features onsupporting information

References [13,2,16–18,3] [14,17,20,21] [15,5,19,10]

business-driven and results-oriented. It often encompasses amixture of tools, databases and vendors, in order to deliver aninfrastructure that not only delivers the initial solution, but alsoincorporatesthe capability of changewith business andthe cur-rent marketplace.

Wu et al. [20] defined BI as a business managementterm used to describe applications and technologies thatare used to gather, provide access to, and analyse data andinformation about the organization to help management makebetter business decisions. In other words, the purpose of BI is to provide business systems with actionable, decision-support technologies, including traditional data warehousingtechnologies, reporting, ad hoc querying and OLAP.

Elbashir et al. [10] refer to BI systems as an important group

of systemsfor dataanalysis andreporting, which supports man-agers at different levels of the organization with timely, rele-vant and trouble-free ways to use information, enabling themto make better decisions. They explain that BI systems are oftenimplemented as enhancements to widely adopted enterprisesystems, such as ERP systems. The scale of investment in BI sys-tems reflects its growing strategic importance, highlighting theneed for more attention in research studies [10].

In some research, BI is concerned with the integration andconsolidation of raw data into key performance indicators(KPIs). KPIs represent an essential basis for business decisionsin the context of process execution. Therefore, operationalprocesses provide the context for data analysis, informationinterpretation, and the appropriate action to be taken [21].

Recently, Jalonen and Lonnqvist [3] wrote that BI generates

analyses and reports on trends in the business environmentand on internal organizational matters. They explained thatanalyses may be produced systematically and regularly, or theymay be ad-hoc, related to a specific decision-making context.Decision makers at different organizational levels employ thisknowledge. The process results in the generation of bothnumerical and textual information.

Two important propositions arise from these definitionsof BI.

1. Often approaches to BI are limited by supported functions,systems or system types.

2. BI is aimed primarily at providing decision-relevant analyticinformation to the management of an organization insupport of their management activities.

In Table 1, BI definitions are sorted based on threeapproaches: a managerial approach a technical approach, andan approach to BI as an enabler of enterprise systems.

In this study, we follow the system-enabler approach to de-fine BI. Actually, organizations would have a better decision-support environment if they were to enhance their enterprisesystems with value-added features and functionalities. Follow-ing is a review of limited efforts in the past to study the evalu-ation of BI in enterprise systems.

Sharma and Djiaw [4], in their managerial study, stated theeffectiveness of Business Intelligence (BI) tools as enablers of knowledge sharing between employees in the organization.They expressed that BI does not stand in isolation fromother initiatives for exploiting knowledge in order to drive

performance, and they concluded that BI tools and capabilitiesare necessary in enterprise systems. Their key message toexecutives was: ‘‘We cannot managewhat we do notmeasure!’’

Lin et al. [11] designed a performance assessment model,and concluded that the accuracy of the output, its conformityto requirements and its support of organizational efficiency arethe most critical factors in gauging the effectiveness of a BIsystem. They set forth the necessity of measurement indicatorsto show the performance of a BI system, but did not provide themeans to evaluate the intelligence of the system.

Lönnqvist and Pirttimäki [5] discussed BI as a set of supportprocesses and stated that most literature focuses on justifyingthe value of BI. This is an important issue when the usefulnessof BI is under initial consideration, and also later when there is

a need to determine if BI continues to provide valuable results.They encouragedpractitioners and researchers to startapplyingthe measurement of BI to their work.

Elbashir et al. [10] developed a new concept, based on anunderstanding of the characteristics of BI systems in a process-oriented framework. They examined the relationship betweenthe performance of business process and organizational perfor-mance, finding significant differences in the strength of theirrelationship in different industrial sectors. They concluded bystressing the need for a better understanding of BI systemsthrough evaluation.

Kahraman et al. [9] discussed the roles of intelligencetechniques in enterprise information systems, to obtain a suc-cessful business strategy. Intelligence techniques are rapidlyemerging as new tools in information management systems.

They stressed that intelligence techniques can be used in thedecision process of enterprise information systems. They con-cluded that hybrid systems that contain two or more intel-ligence techniques would be used more in future; therefore,organizations need to take a sophisticated approach to the eval-uation of the intelligence of their information systems.

Considering recent literature and related work describedabove, organizations need models and approaches to evalu-ate and assess the BI capabilities and competencies of theirwork systems, in order to achieve competitive advantageby making the right decisions at the right time. In this re-search, we have identified the relevant evaluation criteriaand have created an approach to evaluate the intelligenceof enterprise systems. To identify these criteria, in current

research a comprehensive review of relevant literature wasconducted in 2010 and 2011 by authors. Articles from jour-nals, conference proceedings, doctoral dissertations and text-books were identified, analysed, and classified. It was alsonecessary to search through a wide range of studies from dif-ferent disciplines, since numerous criteria are related to the in-telligence of a system and to decision support. Therefore, thescope of the search was not limited to specific journals, confer-ence proceedings, doctoral dissertations and textbooks. Man-agement, IT, computing and IS are some common academicdisciplines in BI research. Consequently, the following online journals, conference databases dissertation databases and text-books were searched to provide a comprehensive bibliogra-phy of the target literature: ABI/INFORM database, ACM DigitalLibrary, Emerald Fulltext, J Stor, IEEE Xplore, ProQuest Digital

7/28/2019 1100215X-main



Table 2: BI evaluation criteria.

Criteria ID Criteria name Related studies

C1 Group sorting tools and methodology (Groupware) [22–25]C2 Group decision making [26–28]C3 Flexible models [23,29,11]C4 Problem clustering [23,30,31]C5 Optimization technique [32–36]

C6 Learning technique [37,19,38,39]C7 Import data from other systems [40,1,34,41]C8 Export reports to other systems [40,42,34]C9 Simulation models [37,34,41,39]C10 Risk simulation [27,43]C11 Financial analyses tools [44–46]C12 Visual graphs [47,48,37,49,35]C13 Summarization [50,51,37,2]C14 Evolutionary prototyping model [52,50,46,53]C15 Dynamic model prototyping [54,50,55–57]C16 Backward and forward reasoning [58,27,53]C17 Knowledge reasoning [40,59,27]C18 Alarms and warnings [2,60,53]C19 Dashboard/recommender [61,62,18]C20 Combination of experiments [63,61,58,64,60,65]C21 Situation awareness modelling [45,59,66]C22 Environmental awareness [67–69]C23 Fuzzy decision-making [70,29,71,72,28]

C24 OLAP [73–75,42,76,77]C25 Data mining techniques [50,42,76,78]C26 Data warehouses [73,79–81]C27 Web channel [73,82,83,2]C28 Mobile channel [2,84,78]C29 E-mail channel [85,86,84]C30 Intelligent agent [46,77,28]C31 Multi agent [87,88,85]C32 MCDM tools [89,90,25,91]C33 Stakeholders’ satisfaction [92,5,27,56]C34 Reliability and accuracy of analysis [93,5,94,29,56,2]

Dissertations, Sage, Science Direct, and Web of Science. The lit-erature search was based on the descriptors, ‘BI capabilities’,‘decision support’, ‘decision-support criteria’, ‘BI evaluation cri-

teria’, ‘BI assessment criteria’, ‘BI requirements’ and ‘intelligenttools capabilities’. The criteria identified are listed in Table 2 asBI evaluation criteria.

3. Research objectives and methodology

To answer the research questions posed in Section 1, severalresearch objectives were determined. The main objective wasto study the effect of BI evaluation of enterprise systems on thedecision-support environments of organizations. Subordinateresearch objectives were to determine the main factors in theevaluation of BI competencies and their relative importance.

Based on a literature review and similar research [95,96], aswell as recent research on BI by the authors, statistical methods

were applied and the research structure was developed basedon the ten stages shown in Figure 1.

The first stage was the literature review of businessintelligence specifications and capabilities, i.e. the criteria of asystem that defines its BI, as listed in Table 2.

In the second stage,a questionnairewas designed withthreemain parts. The first section of the questionnaire consisted of questions about the characteristics of the interviewees. Thecontent of the second section was based on questions about BIcompetency, which were asked to determine the importanceof the evaluation criteria, and finally, the third section of thequestionnaire included questions to learn about the effect of BI evaluation on the decision-support environments of theorganizations.

In the third stage, the survey data from the intervieweeswere collected; to test hypothesis, it was necessary todetermine the statistical distribution of the collected data from

the second part of the questionnaire. Subsequently, based onthe distribution of data, either a parametric or non-parametrictest was performed to prove the hypothesis. The main purposeof the fifth stage was to confirm the hypothesis from stagetwo.

The sixth and seventh stages of the research framework arebased on ‘‘factor analysis’’, and concentrate on the extractionand identification of the BI evaluation criteria affecting theintelligence of enterprise systems. ‘Factor analysis’ is also ageneric name given to a class of multivariate statistical methodswhose primary purpose is to define the underlying structurein a data matrix. Using factor analysis, we first identified theseparate factors of thestructureand then determinedthe extentto which each variable was explained by each factor. Oncethese

factors and the explanation of each variable were determined,summarization and reduction of the data were carried out. Bysummarizing the data, the factor analysis derived underlyingfactors that when interpreted and understood, described thedata in a much smaller number of concepts than the originalindividual variables [97]. Evaluating the suitability of collecteddata, performing factor analysis, and naming the extractedfactors were individual steps.

Finally, the most important factors and their effect weremade clear through labelling. An expert tool was then designedbased on the extracted knowledge of the relationships betweenBI competencies and the main factors of intelligence levels inenterprise systems. This new tool can help an organization tostudy and diagnose the intelligence of its business systems.

7/28/2019 1100215X-main



Figure 1: Research structure and stages.

Table 3: Demographic profiles of interviewees.

Description Number of interviewees

Percent Cumulative(percentage)

GenderMale 154 87.5 87.5Female 22 12.5 100

Sum 176 100

Organization typeGovernmental 102 58 58Private 74 42 100Sum 176 100

Educational degree

Under BS 20 11.4 11.4BS 83 47.2 58.5MS or higher education 73 41.5 100Sum 176 100

Decision-type

Structured 14 8 8Semi structured 55 31.2 39.2Unstructured 107 60.8 100Sum 176 100

Seniority

Less than 5 years 7 4 45 to less than 10 years 69 39.2 43.210 to less than 15 years 64 36.4 79.615 to less than 20 years 25 14.2 93.8

20 years and above 11 6.2 100Sum 176 100

3.1. Design of the questionnaire

A questionnaire was designed and structured in threesections (Appendix). Information related to the basic profileof the interviewees was requested at the beginning of thequestionnaire. In the second part, 34 questions were asked tomeasure their attitudes, based on the importance of the BIevaluationcriteria listedin Table2. The selected responses wereevaluated on a ‘‘Likert Scale’’ [98] and the responses could be:very strongly disagree, strongly disagree, disagree, no opinion,

agree, strongly agree, or very strongly agree. In other words, thesecond part of the questionnaire measures their opinions aboutthe importance of each BI competency of the enterprise system.

Following these 34 questions, one question (Y ) in the thirdpart of the questionnaire was designed to measure the effectof BI evaluation on the decision-support environments of theirorganizations:

Y . Is the evaluation of BI forenterprise systemsimportant to the

decision-support environment in the organization?

3.2. Methodology of the data collection

Following the research objectives, the main targets of thestudy were stakeholders in organizations, who were involved

in decision making and were familiar with BI and IT tools.Therefore, the main targets of the sampling were CIOs (Chief Information Officers), IT Managers, and IT Project Managers,who are involved in IT efforts and decision making. Basedon [99,96], the data-collection method was based on a simplerandomselectionof targets in the list of Fortune 500 companies.

4. Empirical results and analysis

4.1. Data collection

The research targets were CIOs (Chief Information Officers),IT Managers and IT Project Managers. The number of question-naires sent out was 420 and the number returned was 185,which showed a returnrate of 44.04%.Of the returned question-naires, twenty-six were incomplete and thus discarded,makingthe number of valid questionnaires 176, or 41.90% of the totalnumber sent out.

4.2. Demographic profiles of interviewees

The demographic profiles of the interviewees who partici-pated in the survey have been summarized in Table 3. The re-sults show that most participants (87.5%) are male and are from

7/28/2019 1100215X-main



Table 4: Wald-Wolfowitz test results (H1 prove test).

Question Cluster N Numberof runs

Z Asymp. Sig.(1-tailed)

Y 1 15

2 −12.985 0.0002 161

Note: cluster 1 includes very strongly disagree, strongly disagree, disagreeand cluster 2 includes agree, strongly agree or very strongly agree.

both governmental and private organizations. Most of the in-terviewees (88.7%) have a Bachelor of Science (BS) or a higherdegree, as shown in Table 3. On the subject of decision-type, themajority of interviewees make semi-structured and unstruc-tured decisions in their work. Table 3 also shows the seniorityof the participants.As can be seen, 20.4% have over fifteen yearsof seniority, 36.4% have ten to fifteen years, and 43.2% have lessthan ten years seniority.

4.3. Hypothesis test

Inordertoaccomplishthemainobjectiveoftheresearch,the

results should prove the hypothesis. As previously mentioned,one question was posed at the end of the survey that presentedthe hypothesis:H1. Evaluation of the BI of enterprise systemsis vital to the creation

of a decision-support environment in an organization.One of the most popular ways to identify the distribution of

the data, statistically, is the one-sample Kolmogorov–Smirnovtest. The Kolmogorov–Smirnov test compares the observedcumulative distribution function for a variable with a spec-ified theoretical distribution, which may be normal, uni-form, Poisson or exponential. Many statistical parametric testsrequire normally distributed variables. The one-sample Kol-mogorov–Smirnov test canbe used to test whetheror nota vari-able is normally distributed [100]. According to our test results,

the p-valueof allquestions wasless than 0.05, which shows thattheir distribution was abnormal. As the statistical distributionof variablesof Y was also abnormal, a statistical non-parametrictest must be used to prove H 1. For this, a Wald-Wolfowitztest was used to determine the difference between the ‘agree’and ‘disagree’ results. Wald-Wolfowitz is a test that examineswhether two independent samples (clusters) come from thesame population or not. TheWald-Wolfowitztest combines andranks the observations from both groups. If the selected groupsare from the same population, they should be randomly scat-tered throughout the ranking and result in many runs. A signif-icance level below 0.05 indicates that the two groups differ, asalso shown in Table 4.

With reference to a significance-level of less than 0.05 in theWald-Wolfowitz test results, and reaching a consensus for the

main question (Y ), evaluation of the BI of enterprise systemsis vital to the creation of a decision-support environment inan organization. In this way, we can say that an organizationneeds to evaluate the BI specifications of its systems and thisevaluation can improve their decision-support environment.Therefore, from the result of this test of the hypothesis, it canbe concluded that organizations should evaluate their systemswith BI criteria.

4.4. Extraction of factors

Factor analysis is a technique that is mainly suitablefor exploring the patterns of complex, multidimensionalrelationships encountered by researchers. Factor analysis can

Figure 2: Factors of the evaluation model and their loading.

Table 5: Results of the rotated factor analysis.

Factor Initial eigenvalues

Rotation sums of squared loadings

Total Percentage of variance

Cumulativepercentage

1 16.114 7.264 21.366 21.3662 3.173 6.244 18.366 39.7323 1.850 5.044 14.836 54.5684 1.457 2.977 8.755 63.3245 1.348 2.048 6.025 69.3496 1.188 1.553 4.568 73.917

be utilized to examine the underlying patterns or relationshipsfor a large number of variables, and determine whether theinformation can be condensed or summarized into a smaller setof factors or components [97]. An important tool in interpretingfactors is factor rotation. The term rotation means exactly whatit implies. Specifically, the reference axes of the factors areturned about the origin until some other position has beenreached. The un-rotated factor solutions extract factors in theorder of their importance. The first factor tends to be a generalfactor, with almost every variable loading significantly, and itaccounts for the largest amount of variance. The second andsubsequent factors are then based on the residual amounts of variance. The ultimate effect of rotating the factor matrix is toredistribute the variance from earlier factors to later ones toachieve a simpler, theoretically more meaningful factorpattern.The simplest case of rotation is an orthogonal rotation in which

the axes are maintained at 90° [97].In order to determine whether the partial correlation of

the variables is small, the Kaiser–Meyer–Olkin (KMO) test tomeasure sampling adequacy [101] and the Bartlett’s χ2 test of sphericity [102] were used before starting the factor analysis.The result was a KMO of 0.925 and the Bartlett test p-valueless than 0.05, which showed good correlation. The factoranalysis method is the ‘‘principle component analysis’’ in thisresearch, which wasdeveloped by Hotteling [99]. The conditionfor selecting factors was based on the principle proposed byKaiser [101]: ‘‘An Eigen value larger than one, and an absolutevalue of factor loading greater than 0.5’’. The 34 variableswere grouped into six factors and the results can be seen inTable 5. Six factors had an Eigenvalue greaterthan one, and the

7/28/2019 1100215X-main



Figure 3: Functionalities and the first page of the designed tool.

Figure 4: A sample of a questioning step for the diagnosis.

interpretation variable was 73.917%. The factors were rotatedaccording to the Varimax rotation method.

4.5. Naming the factors

The factors were given short labels indicating their content.Based on the meaning and functionalities of the criteria that arerelated to each factor, a conceptual label was assigned to them.The names and content of the six factors are shown in Table 6.The names that have been assigned to the extracted factors are‘‘Analytical and Intelligent Decision-support’’, ‘‘Providing Re-lated Experiment and Integration with Environmental Informa-tion’’, ‘‘Optimization and Recommended Model’’, ‘‘Reasoning’’,‘‘Enhanced Decision-making Tools’’, and finally ‘‘Stakeholders’

Satisfaction’’. Extracted factors and their loading on the mainconcept or BI evaluation are shown in Figure 2. According tothe expectationsof the research [5,11,9,4], the decision-supportcapabilities and BI competencies of enterprise and work sys-tems can be measured and evaluated for each factor. Therefore,there is good conformity between the research objectives andthe results obtained.

5. Designing the BI evaluation tool

The results of the factor analysis indicate that the intelli-gence of enterprise systems can be evaluated based on six mainfactors. To measure the intelligence of these factors, the sys-tem should be evaluated by 34 criteria through questions about

7/28/2019 1100215X-main



Figure 5: Intelligence dashboards and comparison charts of the designed tool.

BI competencies. Using the extracted loads of each criterionwithin its factor, the intelligence of the system can be mea-sured and depicted on six dashboards (for the six factors). Fi-

nally, the most important factors and their effect become clearthrough labelling. The authors have implemented the expertevaluation tool in a VB.NET environment. This tool has threefunctionalities: to evaluate a new system, to edit/view an eval-uated system and to compare evaluated systems. The first pageof this expert evaluating tool is shown in Figure 3. The tool re-quests the systems-type of the enterprise system (ERP, SCM,CRM, Office Automation, etc.) and the organization-type (pri-vate, government or public). It also asks about overall specifi-cations of the enterprise system, such as its functions and thenumber of users. It also requests specifications of the organiza-tion, such as decision-types.

This tool utilizes the 34 BI criteria as questions for diagnosis,and receives answers in a ‘‘Likert Scale’’ format; very low to

veryhigh. A samplepageis shown in Figure4. Computations aredone based on the factors and their loading, on the 34 criteriathat were discussed in Section 4. Reports which include the

overall level of BI and its intelligence level in the six areas aredepicted on the dashboards. A comparison of the factors thatshare in the intelligence of enterprise systems are also shownin charts. A sample of the intelligence-level of a sample systemand its comparison charts are illustrated in Figure 5.

6. Conclusion

This research confirmed the necessity to evaluate BI com-petencies and the specifications of enterprise systems, anddemonstrated that this evaluation can advance decision-support environments. To accomplish this, a survey was carriedout and the responses of the interviewees were grouped intocategories of ‘‘agree’’ (agree, strongly agree, very strongly agree)

7/28/2019 1100215X-main



Table 6: The names and related criteria of factors explored.

Factor Factor name ID. Criteria

F1 Analytical and Intelligent Decision-support

C12 Visual graphsC18 Alarms and warningsC24 OLAPC25 Data mining techniquesC26 Data warehouses

C27 Web channelC28 Mobile channelC30 Intelligent agentC31 Multi agentC13 SummarizationC29 E-mail channel

F2 Providing related experiment and integration with environmental information

C1 Group sorting tools and methodology (Groupware)C3 Flexi ble modelsC4 Problem clusteringC7 Import data from other systemsC8 Export reports to other systemsC20 Combination of experimentsC21 Situation awareness modellingC2 Group decision-makingC22 Environment awareness

F3 Optimization and recommended model

C5 Optimization techniqueC6 Learning technique

C9 Simulation modelsC10 Risk simulationC14 Evolutionary prototyping modelC15 Dynamic model prototypingC19 Dashboard/recommender

F4 ReasoningC11 Financial analyses toolsC16 Backward and forward reasoningC17 Knowledge reasoning

F5 Enhanced Decision-making ToolsC23 Fuzzy decision-makingC32 MCDM tools

F6 Stakeholders’ satisfactionC33 Stakeholders’ satisfactionC34 Reliability and accuracy of analysis

and ‘‘disagree’’(no opinion, disagree, strongly disagree and very

strongly disagree). The Wald-Wolfowitz test was used to de-termine the significant differences between the two groups.The results show that the ‘‘agrees’’ were consistent for the finalquestion (Y ), meaning that organizations should evaluate theBI competencies of systems, which can improve their decision-support environment. From a wide-ranging literature review,34 criteria for BI evaluation were gathered and embedded in thesecond part of the research. The interviewees determined therelative importance of the criteria from these 34 variablesby as-signing ranks to them. The research then applied factor analysisto extract the six main factors for evaluation. These factorswere ‘‘Analytical and Intelligent Decision-support’’, ‘‘Provisionof Related Experimentation and Integration with Environmen-tal Information’’, ‘‘Optimization and Recommendation of aModel’’, ‘‘Reasoning’’, ‘‘Enhanced Decision-making Tools’’, and,finally, ‘‘Stakeholder Satisfaction’’. These are structural factorsthat show the scope of the intelligence of enterprise systemsand their relationship with BI competence. The identificationof these criteria represents an important contribution of thisresearch. Utilizing these criteria, evaluating them in work sys-tems and consequently determining the intelligence of sys-tems can help organizations to improve decision support fordecision makers and enable companies to achieve competitiveadvantage.

Using the results of the factor analysis, the intelligence of enterprise systems can be evaluated according to six mainfactors. To measure the intelligence of these factors, thesystem is evaluated by 34 criteria, using questions aboutBI competencies. Using the extracted loads of each criterion

within its factor, the intelligence of the system is measurable

and is depicted on the six dashboards (for the six factors).An expert tool was designed, which utilizes these criteriato formulate survey questions that reveal the intelligence of each area of an enterprise system. This new tool has thefunctionalities to evaluate a new system, as well as to compareexisting systems, and it produces intelligence-level dashboardsand comparison charts of the intelligence areas.

The authors believe that this research will enable organiza-tions to make better decisions for designing, selecting, evaluat-ing and buying enterprisesystems, using criteria that help themto create a better decision-support environment in their worksystems. The main limitations of this research include the lo-calization of interviewees, differences between the functional-ities of enterprise systems and the novelty of BI in business and

industry. Of course, further research is needed. One importanttopic for the future is the design of expert systems (tools) tocompare vendor products. Another is application of the crite-ria and factors that we have identified and defined in an MCDMframework,in order to select andrank enterprise systemsbasedon BI specifications. The complex relationship between thesefactors and the satisfaction of managers with the decision-making process should also be addressed in future research.

Appendix. The survey instrument

As explained previously a questionnaire as follows has beendesigned and structured in Tables A.1–A.3.

7/28/2019 1100215X-main



Table A.1: Profile of the interviewees.

Gender Male Female

Organization type Governmental Private

Educational degree Under BS BS MS or higher education

Decision-type Structured Semi-structured Unstructured

SeniorityLess than 5 years 5 to less than 10 years 10 to less than 15 years

15 to less than 20 years 20 years and above

Table A.2: Criteria for evaluation of BI.

The following are your attitude about BI capabilities and competencies in enterprise systems

Very stronglydisagree

Stronglydisagree

Disagree No opinion Agree Stronglyagree

Very stronglyagree

Group sorting tools andmethodology (Groupware)

1 2 3 4 5 6 7

Group decision-making 1 2 3 4 5 6 7Flexible models 1 2 3 4 5 6 7Problem clustering 1 2 3 4 5 6 7Optimization technique 1 2 3 4 5 6 7Learning technique 1 2 3 4 5 6 7Import data from other systems 1 2 3 4 5 6 7Export reports to other systems 1 2 3 4 5 6 7

Simulation models 1 2 3 4 5 6 7Risk simulation 1 2 3 4 5 6 7Financial analyses tools 1 2 3 4 5 6 7Visual graphs 1 2 3 4 5 6 7Summarization 1 2 3 4 5 6 7Evolutionary prototyping model 1 2 3 4 5 6 7Dynamic model prototyping 1 2 3 4 5 6 7Backward and forward reasoning 1 2 3 4 5 6 7Knowledge reasoning 1 2 3 4 5 6 7Alarms and warnings 1 2 3 4 5 6 7Dashboard/Recommender 1 2 3 4 5 6 7Combination of experiments 1 2 3 4 5 6 7Situation awareness modelling 1 2 3 4 5 6 7Environmental awareness 1 2 3 4 5 6 7Fuzzy decision-making 1 2 3 4 5 6 7OLAP 1 2 3 4 5 6 7Data mining techniques 1 2 3 4 5 6 7Data warehouses 1 2 3 4 5 6 7Web channel 1 2 3 4 5 6 7Mobile channel 1 2 3 4 5 6 7E-mail channel 1 2 3 4 5 6 7Intelligent agent 1 2 3 4 5 6 7Multi agent 1 2 3 4 5 6 7MCDM tools 1 2 3 4 5 6 7Stakeholders’ satisfaction 1 2 3 4 5 6 7Reliability and accuracy of analysis 1 2 3 4 5 6 7

Table A.3: The necessity of BI evaluation.

Very stronglydisagree

Stronglydisagree

Disagree No opinion Agree Stronglyagree

Very stronglyagree

Y . Is the evaluation of BI, forenterprise systems, important to thepromotion of decision-support inyour organization?

1 2 3 4 5 6 7

References

[1] Alter, S. ‘‘A work system view of DSS in its fourth decade’’, DecisionSupport Systems, 38(3), pp. 319–327 (2004).

[2] Power, D.J. ‘‘Understanding data-driven decision support systems’’,Information Systems Management , 25(2), pp. 149–154 (2008).

[3] Jalonen, H. and Lonnqvist, A. ‘‘Predictive business-fresh initiative or oldwine in a new bottle’’, Management Decision, 47(10), pp. 1595–1609(2009).

[4] Sharma, R.S. and Djiaw, V. ‘‘Realising the strategic impact of businessintelligence tools’’, VINE: The Journal of Information and KnowledgeManagement Systems, 41(2), pp. 113–131 (2011).

[5] Lönnqvist, A. and Pirttimäki, V. ‘‘The measurement of business intelli-gence’’, Information Systems Management , 23(1), pp. 32–40 (2006).

[6] Jadhav, A.S.and Sonar, R.M.‘‘Evaluating and selectingsoftwarepackages:a review’’, Information and Software Technology, 51(3), pp. 555–563(2009).

[7] Jadhav, A.S. and Sonar, R.M. ‘‘Framework for evaluation and selectionof the software packages: a hybrid knowledge based system approach’’, Journal of Systems and Software, doi:10.1016/j.jss.2011.03.034. (2011).

[8] Sen, C.G., Baracli, H., Sen, S. and Basligil, H. ‘‘An integrated decisionsupport system dealing with qualitative and quantitative objectives forenterprise software selection’’, Expert Systems with Applications, 36(3),pp. 5272–5283 (2009).

http://dx.doi.org/doi:10.1016/j.jss.2011.03.034



7/28/2019 1100215X-main



[9] Kahraman, C., Kaya, İ and Çevikcan, E. ‘‘Intelligence decision systems inenterprise information management’’, Journal of Enterprise InformationManagement , 24(4), pp. 360–379 (2011).

[10] Elbashir, M., Collier, P. and Davern, M. ‘‘Measuring the effects of businessintelligencesystems: therelationshipbetweenbusiness processand organizational performance’’, International Journal of Accounting Information Systems, 9(3), pp. 135–153 (2008).

[11] Lin, Y., Tsai, K., Shiang, W., Kuo, T. and Tsai, C. ‘‘Research on using ANPto establish a performance assessment model for business intelligence

systems’’, Expert Systems with Applications, 36(2), pp. 4135–4146 (2009).[12] Nylund, A. ‘‘Tracing the BI family tree’’, Knowledge Management (1999).[13] Ghoshal, S. and Kim, S.K. ‘‘Building effective intelligence systems for

competitive advantage’’, Sloan Management Review, 28(1), pp. 49–58(1986).

[14] Berson, A. and Smith, S., Data Warehousing, Data Mining, and OLAP ,McGraw-Hill, Inc., New York, NY, USA (1997).

[15] Eckerson, Wayne W., Performance Dashboards: Measuring, Monitoring,and Managing Your Business, Wiley (2005).

[16] Maria, F. ‘‘Improving the utilization of external strategic information’’,Master of Science Thesis, Tampere University of Technology (2005).

[17] Petrini,M. andPozzebon, M. ‘‘What roleis businessintelligenceplayingindeveloping countries? a picture of Brazilian companies’’, In Data Mining Applications for Empowering Knowledge Societies, Hakikur Rahman, Ed.,pp. 237–257, IGI Global (2008).

[18] Bose, R. ‘‘Advanced analytics: opportunities and challenges’’, IndustrialManagement & Data Systems, 109(2), pp. 155–172 (2009).

[19] Ranjan, J. ‘‘Business justification with business intelligence’’, The Journalof Information and Knowledge Management Systems, 38(4), pp. 461–475

(2008).[20] Wu, L., Barash, G. and Bartolini, C. ‘‘A service-oriented architecture for

business intelligence’’, IEEE International Conference on Service-OrientedComputing and Applications, SOCA’07, pp. 279–285 (2007).

[21] Bucher, T., Gericke, A. and Sigg,S. ‘‘Process-centric business intelligence’’,Business Process Management Journal, 15(3), pp. 408–429 (2009).

[22] Shim, J.,Warkentin, M.,Courtney, J.,Power, D.,Sharda, R. andCarlsson, C.‘‘Past, present, and future of decision support technology’’, DecisionSupport Systems, 33(2), pp. 111–126 (2002).

[23] Reich, Y. and Kapeliuk, A. ‘‘A framework for organizing the spaceof decision problems with application to solving subjective, context-dependent problems’’, Decision Support Systems, 41(1), pp. 1–19 (2005).

[24] Damart, S., Dias, L. and Mousseau, V. ‘‘Supporting groups in sort-ing decisions: methodology and use of a multi-criteria aggrega-tion/disaggregation DSS’’, DecisionSupport Systems, 43(4),pp. 1464–1475(2007).

[25] Marinoni,O., Higgins,A., Hajkowicz, S. andCollins,K. ‘‘Themultiplecrite-ria analysis tool (MCAT): a new software tool to support environmental

investmentdecisionmaking’’, Environmental Modelling & Software, 24(2),pp. 153–164 (2009).[26] Eom, S. ‘‘Decision support systems research: current state and trends’’,

Industrial Management & Data Systems, 99(5), pp. 213–220 (1999).[27] Evers, M. ‘‘An analysis of the requirements for DSS on integrated

river basin management’’, Management of Environmental Quality: AnInternational Journal, 19(1), pp. 37–53 (2008).

[28] Yu, L., Wang, S. and Lai, K. ‘‘An intelligent-agent-based fuzzy groupdecision making model for financial multicriteria decision support: thecase of credit scoring’’, European Journal of Operational Research, 195(3),pp. 942–959 (2009).

[29] Zack, M. ‘‘The role of decision support systems in an indeterminateworld’’, Decision Support Systems, 43(4), pp. 1664–1674 (2007).

[30] Loebbecke, C. and Huyskens, C. ‘‘Development of a model-based netsourcing decision support system using a five-stage methodology’’,European Journal of Operational Research, 195(3), pp. 653–661 (2009).

[31] Lamptey, G., Labi, S. and Li, Z. ‘‘Decision support for optimal schedulingof highway pavement preventivemaintenancewithin resurfacingcycle’’,Decision Support Systems, 46(1), pp. 376–387 (2008).

[32] Lee,J. and Park,S. ‘‘Intelligent profitablecustomerssegmentationsystembased on business intelligence tools’’, Expert Systems with Applications,29(1), pp. 145–152 (2005).

[33] Nie, G., Zhang, L., Liu, Y., Zheng, X. and Shi, Y. ‘‘Decision analysis of datamining project based on Bayesian risk’’, Expert Systems with Applications,36(3), pp. 4589–4594 (2009).

[34] Shang, J., Tadikamalla, P., Kirsch, L. and Brown, L. ‘‘A decision supportsystem for managing inventory at GlaxoSmithKline’’, Decision Support Systems, 46(1), pp. 1–13 (2008).

[35] Azadivar, F., Truong, T. and Jiao, Y. ‘‘A decision support system forfisheries management using operations research and systems scienceapproach’’, Expert Systemswith Applications,36(2),pp.2971–2978(2009).

[36] Delorme, X., Gandibleux, X. and Rodríguez, J. ‘‘Stability evaluation of a railway timetable at station level’’, European Journal of OperationalResearch, 195(3), pp. 780–790 (2009).

[37] Power, D. and Sharda, R. ‘‘Model-driven decision support systems:concepts and research directions’’, Decision Support Systems, 43(3),pp. 1044–1061 (2007).

[38] Li, D., Lin, Y. and Huang, Y. ‘‘Constructing marketing decision supportsystems using data diffusion technology: a case study of gas stationdiversification’’, Expert Systems with Applications, 36(2), pp. 2525–2533(2009).

[39] Zhan, J., Loh, H.T. and Liu, Y. ‘‘Gather customer concerns from onlineproduct reviews—a text summarization approach’’, Expert Systems with Applications, 36(2), pp. 2107–2115 (2009).

[40] Ozbayrak, M. and Bell, R. ‘‘A knowledge-based decision support systemforthe management of parts andtoolsin FMS’’, Decision Support Systems,

35(4), pp. 487–515 (2003).[41] Quinn, N.W.T. ‘‘Environmental decision support system development forseasonal wetland salt management in a river basin subjected to waterquality regulation’’, Agricultural Water Management , 96(2), pp. 247–254(2009).

[42] Shi, Z., Huang, Y., He, Q., Xu, L., Liu, S., Qin, L., Jia, Z., Li, J., Huang, H. andZhao, L. ‘‘MSMiner—a developing platform for OLAP’’, Decision Support Systems, 42(4), pp. 2016–2028 (2007).

[43] Galasso, F. and Thierry, C. ‘‘Design of cooperative processes in acustomer-supplier relationship: an approach based on simulation anddecision theory’’, Engineering Applications of Artificial Intelligence, 22(6),pp. 865–881 (2008).

[44] Santhanam, R. and Guimaraes, T. ‘‘Assessing the quality of institutionalDSS’’, European Journal of Information Systems, 4(3), pp. 159–170 (1995).

[45] Raggad, B.G. ‘‘Decision support system: use IT or skip IT’’, IndustrialManagement & Data Systems, 97(2), pp. 43–50 (1997).

[46] Gao, S. and Xu, D. ‘‘Conceptual modeling and development of anintelligent agent-assisted decision support system for anti-moneylaundering’’, Expert Systems with Applications, 36(2), pp. 1493–1504

(2009).[47] Noori, B. and Salimi, M.H. ‘‘A decision-support system for business-to-business marketing’’, Journal of Business & Industrial Marketing , 20(4–5),pp. 226–236 (2005).

[48] Kwon, O., Kim, K. and Lee, K.C. ‘‘MM-DSS: integrating multimedia anddecision-makingknowledge in decisionsupportsystems’’, ExpertSystemswith Applications, 32(2), pp. 441–457 (2007).

[49] Li, S., Shue, L. and Lee, S. ‘‘Business intelligence approach to supportingstrategy-making of ISP service management’’, Expert Systems with Applications, 35(3), pp. 739–754 (2008).

[50] Bolloju, N., Khalifa, M. and Turban, E. ‘‘Integrating knowledge manage-mentinto enterpriseenvironmentsfor the nextgeneration decision sup-port’’, Decision Support Systems, 33(2), pp. 163–176 (2002).

[51] Hemsley-Brown, J. ‘‘Using research to support management decisionmaking within the field of education’’, Management Decision, 43(5),pp. 691–705 (2005).

[52] Fazlollahi, B. and Vahidov, R. ‘‘Extending the effectiveness of simulation-based DSS through genetic algorithms’’, Information & Management ,39(1), pp. 53–65 (2001).

[53] Xiaoshuan, Z., Zetian, F., Wengui, C., Dong, T. and Jian, Z. ‘‘Applyingevolutionary prototyping model in developing FIDSS: an intelligentdecision support system for fish disease/health management’’, Expert Systems with Applications, 36(2), pp. 3901–3913 (2009).

[54] Koutsoukis, N., Dominguez-Ballesteros, B., Lucas, C.A. and Mitra, G.‘‘A prototype decision support system for strategic planning underuncertainty’’, International Journal of Physical Distribution & LogisticsManagement , 30(7–8), pp. 640–660 (2000).

[55] Goul, M. and Corral, K. ‘‘Enterprise model management and next gen-eration decision support’’, Decision Support Systems, 43(3), pp. 915–932(2007).

[56] González,J.R., Pelta, D.A.and Masegosa,A.D. ‘‘Aframework fordevelopingoptimization-based decision support systems’’, Expert Systems with Applications, 36(3), pp. 4581–4588 (2008).

[57] Pitty, S., Li, W., Adhitya, A., Srinivasan, R. and Karimi, I.A. ‘‘Decision sup-port for integrated refinery supply chains Part 1. Dynamic simulation’’,Computers and Chemical Engineering , 32(11), pp. 2767–2786 (2008).

[58] Gottschalk, P. ‘‘Expert systems at stageIV of the knowledgemanagementtechnologystage model: thecase of policeinvestigations’’,Expert Systems

with Applications, 31(3), pp. 617–628 (2006).[59] du Plessis,T. and du Toit,A. ‘‘Knowledge managementand legalpractice’’,

International Journal of Information Management , 26(5), pp. 360–371(2006).

[60] Ross, J.J., Denaı, M.A. and Mahfouf, M. ‘‘A hybrid hierarchical decisionsupport system forcardiac surgical intensivecare patients.Part II.Clinicalimplementation and evaluation’’, Artificial Intelligence in Medicine, 45(1),pp. 53–62 (2008).

[61] Nemati, H.R., Steiger, D.M. and Iyer, L., et al. ‘‘Knowledge warehouse: anarchitectural integration of knowledge management, decision support,artificial intelligence and data warehousing’’, Decision Support Systems,33(2), pp. 143–161 (2002).

[62] Hedgebeth, D. ‘‘Data-driven decision making for the enterprise: anoverviewof businessintelligenceapplications’’,The Journal of Informationand Knowledge Management Systems, 37(4), pp. 414–420 (2007).

[63] Courtney, J.F.‘‘Decision making and knowledgemanagementin inquiringorganizations: toward a new decision-making paradigm for DSS’’,Decision Support Systems, 31(1), pp. 17–38 (2001).

7/28/2019 1100215X-main



[64] Gonnet, S., Henning, G. and Leone, H. ‘‘A model for capturing andrepresenting the engineering design process’’, Expert Systems with Applications, 33(4), pp. 881–902 (2007).

[65] Hewett, C., Quinn, P., Heathwaite, A.L., Doyle, A., Burke, S., Whitehead, P.and Lerner, D. ‘‘A multi-scale framework for strategic management of diffuse pollution’’, Environmental Modelling & Software, 24(1), pp. 74–85(2009).

[66] Feng, Y.H., Teng, T.H. and Tan, A. ‘‘Modelling situation awareness forcontext-aware decision support’’, ExpertSystems withApplications, 36(1),

pp. 455–463 (2009).[67] Phillips-Wren, G., Hahn, E. and Forgionne, G. ‘‘A multiple-criteriaframework for evaluation of decision support systems’’, Omega, 32(4),pp. 323–332 (2004).

[68] Koo, L.Y., Adhitya, A., Srinivasan, R. and Karimi, I.A. ‘‘Decision supportfor integrated refinery supply chains. Part 2. Design and operation’’,Computers and Chemical Engineering , 32(11), pp. 2787–2800 (2008).

[69] Calof, J.L. and Wright, S. ‘‘Competitive intelligence: a practitioner, aca-demicand inter-disciplinary perspective’’, European Journal of Marketing ,42(7–8), pp. 717–730 (2008).

[70] Metaxiotis, K., Psarras, J. and Samouilidis, E. ‘‘Integrating fuzzy logicinto decision support systems: current research and future prospects’’,Information Management & Computer Security, 11(2), pp. 53–59 (2003).

[71] Makropoulos, C.K., Natsis, K., Liu, S., Mittas, K. and Butler, D. ‘‘Decisionsupport for sustainable option selection in integrated urban water man-agement’’, Environmental Modelling & Software, 23(12), pp. 1448–1460(2008).

[72] Wadhwa, S., Madaan, J. and Chan, F.T.S. ‘‘Flexible decision modelingof reverse logistics system: a value adding MCDM approach for

alternative selection’’, Robotics and Computer-Integrated Manufacturing ,25(2), pp. 460–469 (2009).[73] Tan, X., Yen, D. and Fang, X. ‘‘Web warehousing: web technology meets

data warehousing’’, Technology in Society, 25(1), pp. 131–148 (2003).[74] Lau,H.C.W.,Ning, A.,Ip,W.H.and Choy,K.L.‘‘A decisionsupportsystem to

facilitateresources allocation: an OLAP-basedneuralnetworkapproach’’, Journal of Manufacturing Technology Management , 15(8), pp. 771–778(2004).

[75] Rivest, S., Bédard, Y., Proulx, M., Nadeau, M., Hubert, F. and Pastor, J.‘‘SOLAP technology: merging business intelligence with geospatialtechnology for interactive spatio-temporal exploration and analysis of data’’, ISPRSJournalof Photogrammetry& Remote Sensing ,60(1),pp.17–33(2005).

[76] Berzal, F., Cubero, J. and Jiménez, A. ‘‘The design and use of theTMiner component-based data mining framework’’, Expert Systems with Applications, 36(4), pp. 7882–7887 (2008).

[77] Lee, C.K.M., Lau, H.C.W., Hob, G.T.S.and Ho,W. ‘‘Design anddevelopmentof agent-based procurement system to enhance business intelligence’’,Expert Systems with Applications, 36(1), pp. 877–884 (2009).

[78] Cheng, H., Lu, Y. and Sheu, C. ‘‘An ontology-based business intelligenceapplication in a financial knowledge management system’’, Expert Systems with Applications, 36(2), pp. 3614–3622 (2009).

[79] Tseng, F.S.C. and Chou, A.Y.H. ‘‘The concept of document warehousingfor multi-dimensional modeling of textual-based business intelligence’’,Decision Support Systems, 42(2), pp. 727–744 (2006).

[80] March, S.T. and Hevner, A.R. ‘‘Integrated decision support systems:a data warehousing perspective’’, Decision Support Systems, 43(3),pp. 1031–1043 (2007).

[81] Manh Nguyen, T., Min Tjoa, A., Nemec, J. and Windisch, M. ‘‘An approachtowards an event-fed solution for slowly changing dimensions in datawarehouses with a detailed case study’’, Data & Knowledge Engineering ,63(1), pp. 26–43 (2007).

[82] Oppong, S.A., Yen, D.C. and Merhout, J.W. ‘‘A new strategy for harnessingknowledge management in e-commerce’’, Technology in Society, 27(3),pp. 413–435 (2005).

[83] Anderson, J.L., Jolly, L.D. and Fairhurst, A.E. ‘‘Customer relationshipmanagement in retailing: a content analysis of retail trade journals’’,

Journal of Retailing and Consumer Services, 14(6), pp. 394–399 (2007).[84] Wen, W., Chen, Y.H. and Pao, H.H. ‘‘A mobile knowledge managementdecision support system for automatically conducting an electronicbusiness’’, Knowledge-Based Systems, 21(7), pp. 540–550 (2008).

[85] Granebring,A. and Re’vay, P. ‘‘Service-orientedarchitectureis a driverfordaily decision support’’, Kybernetes, 36(5–6), pp. 622–635 (2007).

[86] Baars, H. and Kemper, H. ‘‘Management support with structured andunstructured data—an integrated business intelligence framework’’,Information Systems Management , 25(2), pp. 132–148 (2008).

[87] Bui, T. and Lee, J. ‘‘An agent-based framework for building decisionsupport systems’’, Decision Support Systems, 25(3), pp. 25–237 (1999).

[88] Xu, D. and Wang, H. ‘‘Multi-agent collaboration for B2B workflowmonitoring’’, Knowledge-Based Systems, 15(8), pp. 485–491 (2002).

[89] Hung,S.Y.,Ku,Y.C.,Liang, T.P. andLee,C.J.‘‘Regret avoidanceas a measureof DSS success: an exploratory study’’, Decision Support Systems, 42(4),

pp. 2093–2106 (2007).[90] Yang, I.T. ‘‘Utility-based decision support system for schedule optimiza-tion’’, Decision Support Systems, 44(3), pp. 595–605 (2008).

[91] I, Y.T. and Yurdakul, M. ‘‘Development of a decision support systemfor machining center selection’’, Expert Systems with Applications, 36(2),pp. 3505–3513 (2009).

[92] Goodhuea, D.L., Kleinb, B.D. and March, S.T. ‘‘User evaluations of IS assurrogates for objective performance’’, Information& Management , 38(2),pp. 87–101 (2000).

[93] Gregg, D.G., Goul, M. and Philippakis, A. ‘‘Distributing decision supportsystems on the WWW: the verification of a DSS metadata model’’,Decision Support Systems, 32(3), pp. 233–245 (2002).

[94] Phillips-Wren, G., Mora, M., Forgionne, G. and Gupta, J. ‘‘An integrativeevaluation framework for intelligent decision support systems’’, Euro- pean Journal of Operational Research, 195(3), pp. 642–652 (2009).

[95] Jafari, M., Akhavan, P. and Rezaee Nour, J. ‘‘Knowledge management inIranaerospaceindustries: a studyon critical factors’’, Aircraft Engineering and Aerospace Technology. An International Journal, 79(4), pp. 375–389(2007).

[96] Cudeck, R. and MacCallum, R.C., Factor Analysis at 100 HistoricalDevelopments and Future Directions, LawrenceErlbaum Associates(2007).

[97] Hair, J.F., Anderson, R.E., Tatham, R.L. and Black, W.C., Multivariate Data Analysis, Prentice-Hall, Upper Saddle River, NJ pp. 7–232 (1998).

[98] Likert, R. ‘‘The method of constructing an attitude scale’’, In Scaling: ASourcebook for Behavioral Scientists, G.M. Maranell, Ed., pp. 21–43, AldinePublishing Company, Chicago, IL (1974).

[99] Hotteling, H. ‘‘The most predictable criterion’’, Journal of EducationalPsychology, 26, pp. 139–142 (1935).

[100] Hollander, M. and Wolfe, D.A., Nonparametric Statistical Methods, Wiley,New York, NY pp. 21–132 (1973).

[101] Kaiser, H. ‘‘The varimax criterion for analytic rotation in factor analysis’’,Psychometrika, 23(3), pp. 187–200 (1958).

[102] Bartlett, M.S. ‘‘Test of significance in factor analysis’’, British Journal of Psychology, 3, pp. 77–85 (1950).

Mehdi Ghazanfari is full Professor in the Industrial Engineering Department at

Iran University of Science and Technology (IUST). He received his Ph.D. degreein System Analysis and Production Planning from NSW University, Australia, in1996.He is currentlyEditor-in-Chiefof theNew Economyand Commerce(JNEC)

Journal, and has had more than 30 papers published in the areas of data mining,artificial intelligence and information systems.

Mostafa Jafari has a B.E. degree in Mechanical Engineering, a M.E. degree inProductivity Engineering and a Ph.D. degree in Industrial Engineering fromIIT, Delhi. He is currently Assistant Professor in the Industrial EngineeringDepartment at Iran University of Science and Technology (IUST), Tehran, Iran,working in areas of strategic planning, business process reengineering, andknowledge management. He has had more than 20 research papers and fivebooks published in the field of Industrial Engineering.

Saeed Rouhani is a Ph.D. degree candidate of System Engineering at IranUniversityof Scienceand Technology, Tehran, Iran. He receivedhis B.S.degreein

IndustrialEngineering, in 2003,from IranUniversity of Science and Technology,Tehran, Iran, and a MA degree in Information Technology Management, in2005, from Allame Tabatabiee University, Tehran, Iran. His research interestsare enterprise resource planning systems, business intelligence, informationtechnology, and decision making. He has had three books and more than 10papers published in different conferences and journals.

Documents

1100215X-main