Embed Size (px)
Citation preview
University of Calgary
PRISM: University of Calgary's Digital Repository
Graduate Studies Legacy Theses
1998
An artificial neural network approach to assess
project cost and time risks at the front end of
projects
Liu, Xiaoying
Liu, X. (1998). An artificial neural network approach to assess project cost and time risks at the
front end of projects (Unpublished master's thesis). University of Calgary, Calgary, AB.
doi:10.11575/PRISM/23290
http://hdl.handle.net/1880/42597
master thesis
University of Calgary graduate students retain copyright ownership and moral rights for their
thesis. You may use this material in any way that is permitted by the Copyright Act or through
licensing that has been assigned to the document. For uses that are not allowable under
copyright legislation or licensing, you are required to seek permission.
Downloaded from PRISM: https://prism.ucalgary.ca
THE UNIVERSITY OF CALGARY
An Artificial Neural Network Approach
to Assess Project Cost and Tirne Risks at The Front End of Projects
Xiaoying Liu
A THESIS
SUBMITTIED TO THE FACULTY OF GRADUATE STUDIES
IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE
DEGREE OF MASTER OF SCIENCE
DEPARTMENT OF CIVIL ENGINEERING
CALGARY,ALBERTA
APRIL, 1998
1998 O Xiaoying Liu
National Cibrary of Canada
BiMiothèque nationale du Canada
Acquisitions and Acquisitions et Bibliographie Services seMces bibliographiques 385 Wetihgtm Street 395, ru8 Wellington OriawaON K 1 A W -WB ON K1A ON4 Canada canada
The author has granted a non- exciusive licence allowing the National Libmry of Caoada to reproduce, loan, distn'bute or sell copies of this thesis in rnicroform, paper or electronic formais.
The author retains ownefship of the copyright in this thesis. Neither the thesis nor substantial extracts fiom it may be printed or otherwise reproduced without the author's ~ermission.
L'auteur a accordé une licence non exclusive permettant à la Bibliothèque nationale du Canada de reproduire, prêter, distniuer ou vendre des copies de cette thèse sous la fonne de microfiche/nlm, de reproduction sur papier ou sur format électronique.
L'auteur conserve la propriété du droit d'auteur qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son
The objectives of this research were to explore the applications of artificial neural
network technology in capital project risk analysis and to develop intelligent cornputer
models to predict project cost and Ume variations at the front end stages of projects. The
models were used to evaluate the potential effects of nsks and project decisions on
outcomes. The application of this technology will help decision makea improve the
effectiveness of the decision making process at the front-end stage of projects in the oil
and gas industry.
Results indicated that artificial neural networks have the capability to capture general
patterns by learning from samples of similar past projects. Artificial neural network
models were superior to multiple linear regression models in the prediction of project cost
and thne variations. Artificial neural networks with stepwise regression provide more
accurate estimations than stand alone neural networks. The research showed that
artificial neural network technology has promising potential in the domain of project risk
analysis.
The procedures for developing artincial neural network models to predict project cost and
time variances were proposed. This research provides valuable results and
recommendations for fiiture research work in this area.
ACKNOWLEDGEMENT
This thesis was made possible through the contributions of many people in a variety of
ways. First, the author would like to thank al1 survey respondents for their cooperation.
As well, great thanks to Keith Pedwell, Wynne Chin, Ben Magnusson, Bob McTague,
Richard Balfour, Peter Bourque, Doug Rowan, Tony Goldsmith and Lome Berg for their
advice and suggestions.
With sincerity, the author would like to thank her s u p e ~ s o r , George lergeas, for his
ongoing patience and support during this entire process. Special thanks also to Francis
Hartman for his valuable suggestions, guidance with and sponsorship of this work.
The author wishes to extend her gratitude to Jackie Wilson, Daji Gong, Erin Williamson,
and Greg Skulmoski for helping to edit and review the thesis.
To my father, a great father and teacher, and my mother
TABLE OF CONTENTS
TABLE OF CONTENTS .................................................................................................. V I
......................................................................................... CHAPTER 1 INTRODUCTION 1
.............................................................................................................. BACKGROUND 1
.................................................................................... ARTIFICIAL NEURAL NETWORK 3
.................................................. RISK ANALYSIS AND ARTIFICIAL NEURAL NETWORK 4
P ~ C I P A L OBJECTIVES OF THE RESEARCH ................................................................... 5
.......................................................................................... RESEARCH METHODOLOGY 6
....................................................................................................................... PROCESS 9
PRINCIPAL ACHIEVEMENTS .......................................................................................... 9
................................................................................................. GUIDE TO THE THESIS 12
CEAPTER 2 REVIEW OF PROJECT RISK ANALYSIS AND TECEINIQUES ....... 14
.......................... 2.3. CU- USAGE AND BENEFITS OF ~ S K ANALYSE M INDUSTRY 16
............................................... 2.4. PROJEC~ ESTIMATE AND CONTMGENCY ALLOWANCE 19
................................................................... 2.5. REVIEW OF BSK ANALYSIS TECHNIQUES 21
.................................................. 2.6. DATA REQUIRED FOR PERFORMMG RISK ANALYSE 33
........................................ . 3.1 INTRODUCTION TO ARTIFICIAL NEURAL NEWORK (ANN) 35
.................... ........................ 3.2. ARTIFICIAL NEURAL NETWORK TECHNOLOGY ......... 3 5
....................................... 3.2. 1 . Hktorical Background of Arttpcial Neural Network 35
...................................................................... 3.2.2. Artifcial Neural Network Model 3 7
3.3. COMPARISON BETWEEN ANN's AND OTHER TECHNIQUES ......................................... 42
.................................................. . R3.1. Amifcial Neural N e ~ k vs Expert System 4 2
....................................... 3.3.2. Neural Networks vs . Conventional Stamic AnaiysrS 46
.................................................... 3.4. ISSUES RELATED TO NEURAL NETWORK TRAMMG 47
3.5. APPLICATIONS OF ARTIFICML NEURAL NETWORK M BUSINESS AND NAGEME MENT.^^
3.6. POTENTIAL APPLICATIONS OF NEURAL NETWORK M PROECT WSK ANALYSIS ......... 51
................................................................................................. 4.0. LITERATURE SEARCH 53
4.1. INDUSTRY SURVEY ..................................................................................................... 54
....................................................................................................... 4.1.0. Introduction 5 4
.................................................... Research Advkory Commilee in Indusfry 5 5
.................................................................. Ethics Approvol on Human Studies 55
...................................................................................... Expert Panel Interview 5 6
................................................... Design and Testing of Survey Questionnaire 5 7
.................................................................................................... Samp le Design 59
............................................................................ Administration of the Survey 6 1
......................................................................................... Raw Data Anabsis 6 2
................. 4.2. DEVELOPMENT OF ARTIFICIAL NEURAL NETWORK MODELS 62
.............................................................................. 4.2.1. Introduction to ANN Model 6 2
............................................ 4.2.1.1. Artificial Neural Network Learning Algorithm 63
............................................................................ 4.2.1.2. Layers and Neuron Nodes -69
....................................................................................... 4.2.1 . 3 . Transfer Functions -71
............................................... 4.2.2. Artricial Neural Network Training Procedures 74
........................................................................ 4.2.3. Issues Related to AMV Training 7.5
4.2.3.1. Leaming Rate, Momentum Constant and Training Tolerance ................... -75
4.2.3 .2 . Training and Testing Examples ............................................................. 7 7
4.2.4. Perjiormance Measures of Neural Network ModeLF ........................................ 78
............................................................... 4.2.5. Generalization and Cross- Validation 80
........................................ 4.3. NEURAL NETWORK COMPUTATION SOFTWARE PACKAGES 81
CHAPTER 5 STUDY RESULTS ~ ~ a ~ ~ ~ a ~ ~ a m ~ ~ m a a a ~ m ~ ~ ~ m ~ ~ ~ ~ ~ a ~ ~ m ~ o ~ m a a m m m m m ~ a a m m ~ ~ ~ ~ ~ ~ ~ ~ m ~ m m ~ o a ~ ~ a ~ m ~ a ~ e a ~ a ~ a m m a m m a ~ ~ ~ 8 3
5.1. RAW DATA ANALYSIS ............................. ,., ................................................................ 83
........................................ 5 .2 . ARTIFICIAL NEURAL NETWORK TRAMMG AND RESULTS 8 5
.............................................................. 5.2. I . Formatring of Input Data in Training 87
........................................................................................ S . 2.2. Preliininary Training 8 7
. . ............................................................................................. 5.2.3. Phase I Training 8 9
..................................................... 5.2.3.1. Training and Results with 60% Samples -90
...................................................... 5.2.3.2. Training and Results with 75% Samples 92
..................................... 5.2.3.3. Training and Results with 90% Samples
...................................................... 5.2.4. Comparison arnung Three Group Samples 9 6
5.2. 5 . Defining Critical Input Variables Using Multiple Linear Regression ............. 97
..................................................................................... . 5.2.5.1 Forward Regression -99
5.2.5.2. Forward Regression with Dummy Variables ......................................... 100
..................................... 5.2.5.3. Forward Regression without Durnmy Variables 1 05
....................................... 5.2.6. Phase II Training - Using Critical Input Variables 107
5.3. PERFORMANCE COMPARISON BETWEEN NEURAL NETWORK AND MULTIPLE LMEAR~ 1 1
REGRESSION ANALYSIS .................................................................................................... 1 11
CHAPTER 6 CONCLUSIONS & RECOMMENDATIONS ................... .................... 1 12
APPENDIX V, A SAMPLE OF A BATCH FILE FOR ANN TRAINING ..... , ......... 151
CHAPTER 1 INTRODUCTION
1.1. Background
At the early stages of a project, decision making relies on experienced practitioners.
These early decisions are made mostly at the time when there are a great amount of
unceaainties involved. The quaiity of decisions cannot be venfied until the project is
completed. Artificial intelligence based decision support models are one option to help
practitioners screen alternative decisions and evaluate their impact on project outcornes in
ternis of project fuial cost and duration, thereby improving the quality of early decisions
and the effectiveness of decision making.
To test this concept, this thesis describes the development of some artificial intelligence
models using artificial neural networks to predict project cost and tirne variances.
Decisions at the fiont-end stage of a project are inherent with uncertainties and risks.
New technology, increasing complexity, political involvement, a changing economy and
regulatory environment, weatherfnatural conditions are some of the major sources of
uncertainty and risk. A contingency is usually allocated to allow for the uncertainties in a
project. Based on i n t e ~ e w s with practitioners, the researcher observed that the
conventional methods of contingency provision, for instance, the use of a flat 'percentage
rate' and 'classes of estimate', are popularly used. Since the contingency allocation is
largely a matter of judgrnent and, therefore, arbitrary, estimators often Fud their estimates
difficult to justify or defend [Dey, Tabucanon and Ogunlana, 19941. The needs for
quantitative risk analysis to provide a rational bais for estimating and contingency
allocation have been addressed in the literature [Cooper and Chapman, 19871. Personal
interviews with practitioners in industry led to the conclusion that the need for better
techniques is increasing in industry as the project environment is becoming more
complex.
Conventionai risk analysis techniques, such as Monte Car10 analysis, provide tools to
help practitioners to assess impacts of uncertainties, to support the detemination or the
assessrnent of the risk level of a project, and to allocate a contingency associated with the
possibility of success. Unfortunately, the effectiveness of using this technique is heavily
dependent upon experts' opinions and judgments. These experts usually represent the
key funftions required to support a project, such as marketing, finance, estimating,
planning, engineering/design, procurement, construction and operation. To capture the
experts' knowledge and opinions is tirne consuming, laborious and complex.
hdeed some risk analysis techniques have other weaknesses, an important one being the
inability to quant@ correlations and interactions between risk factors and to mode1 a
complex system effectively.
1.2. Artificial Neural Network
In cornparison to conventional risk analysis techniques, an Artificial Neural Network
(ANN) is an approach that is fiee of mathematical models. It requires less expert opinion
and judgments than do other techniques. Neural networks represent an attempt to
simulate the human brain's learning process through massive training. A neural network
is able to leam fiom samples. Knowledge learned is stored within the network. This
technology provides a powerful and robust means to assess uncertainty through learning
and capturing general patterns in available data.
Amficial neural network technology is used in many areas ranging from engineering to
business management, especially in finance and banking management. One of the most
significant feanires of a multiple-layer neural network is its generalization and
classification capabilities. These capabilities are utilized to develop predictive models in
real hancing and banking applications.
The application of neural network technology is rare in project cost and time risk
analysis. The one example found in the literatue is McKim's [1993] development of a
neural network model to predict project cost o v e m . The predictive error of the model
was less than half compared to using a conventional method - arithmetical mean.
McKim's study showed that the ne& network approach outperformed the conventional
4
method. The initial research work in this area considered only four risk factors and only
twenty project samples were used to develop the ANN model.
Cross-validation (e.g., R2 in testing data which are unknown to the model) is an important
performance measure of predictive models for both traditionai method models and ANN
models. Unfortunately, it was not performed by the authors (McKim [1993], Denton
[1995], Sohl [1995]) in the development of their ANN models. This may Ieaves the
models developed both unreliable and invalidated.
This thesis attempts to consider more risk factors than other researchea in this area in the
development of ANN models to predict both cost variations and t h e variations. Cross-
validation is applied as a major performance measure in evduating the ANN models.
This thesis is intended to make some progress in this research area.
1.3. Risk Analysis and Artifical Neural Network
Project risk analysis basically involves the identification of uncertainties and the
assessrnent of the overali effects of these uncertainties. In this thesis, risk impacts on the
overall project cost and duration were assessed using artifîcial neural network technology.
The sources causing project cost and t h e variations were identified by the author as
falling under two major categories. The h t category has to do with issues that relate to
the nature of the project, such as, the type of project, location, and complexity. n i e
second category identifies project decision related issues such as fast-tracking approach,
contract strategy, etc.
Al1 these attributes have different degrees of efEects on project cost and time relative to
the original estimates. In other words, these factors have certain correlations with project
cost and time variances. Artificiai neural network models are able to map the
relationships between these factors and the van-ances of cost and duration by 1e-g
extensively fiom real project samples and capturing general patterns within these
samples. Trained neural networks can therefore be used as intelligent predictive models.
7.4. Principal Objectives of the Research
The main objectives of the research are to explore artincial neural network applications in
project risk anaiysis, and to develop intelligent predictive models for project cost and
time variances at early stages in the development of projects. The models will form the
basis for the development of an estimation decision support system. The decision support
system wiil aiso help ownea evaluate impacts of rkks and decisions on project cost and
6
tirne in the early stage of project, thus enabling improved evaluation of alternatives when
making critical formative decisions.
The secondary objective of the research was to identiQ major risk factors that have
significant impacts on project cost and time variations. This information was then to be
used to develop efficient and effective artificial neural network models.
1.5. Research Methodology
The stages of the research methodology followed were:
1. Research Proposal
An intensive search and review of the available literature was perforrned in the areas of
risk analysis and artificid neural network applications in project and business
management. Knowledge of the limitations of existing risk anaiysis techniques and the
unique advantages of artificial neural network technology was gained and the research
objectives were identified. (See Chapter 2 & 3)
2. Data Collection and Suweys
Having identified the research objectives, the author carried out the following activities in
the second stage:
1. Established a industrial research cornmittee consisting of four people fiom the oil and
gas industry to direct the research, to help the researcher identify project critical risk
factors and to provide expertise and references. (See Chapter 4)
2. Applied for ethics approval on human studies. (See Chapter 4)
3. Conducted an exploratory study to identiQ major project risk factors by interviewhg
eight experts who had rich and extensive experience in nsk and project management.
Results fiorn the exploratory study were used to form a structured questionnaire. (See
Chapter 4)
4. Conducted a s w e y of thirty-one practitioners in project management to investigate
the current usage and benefits of using risk analysis techniques by industry. The
s w e y was completed by mail or through personal interviews wherever possible.
(See Chapter 2)
8
5. Designed a structured questionnaire and tested it with colleagues and practitioners.
(See Chapter 4)
6. Carried out an industry survey through personal i n t e ~ e w and mailing within twenty-
four organizations including owner organizations and EPC consulting companies.
Data from one hundred and six completed projects were coilected. The types of
projects primarily included pipeline, refmery, and cornpressor station projects. Al1 of
the data collected referred to projects that were completed within the last 5 years and
had a value of over $1 million. (See Chapter 4)
7. Performed a descriptive analysis of the raw data gathered fiom the survey.
Information collected from the sample projects included the standard deviation and
average values of cost and tirne overnins. (See Chapter 5)
3. Development of ANN Models
ANN technology was applied to develop intelligent models using the survey data. A
three-layer neural network with back-propagation Ieaming algorithm was used. Transfer
fiinctions of the hidden and output layers, the learning rate, the number of hidden nodes,
and the training tolermce were determined through training processes for the effective
performance of the models. (See Chapter 4)
9
The ANN mode1 development consisted of three phases. These are preliminary training,
phase 1 training and phase II training. In the preliminary training, a training d e and a
learning rate were selected; transfer fûnctions on hidden and output layers were initialiy
screened. In phase 1, the h?iasfer fiinctions were fmally determined. These transfer
functions allow ANN models to reach the best performance in generalization capability.
In phase II, ANN models were developed using cntical input variables and three groups
of project data. A forward regression technique was used to define the critical input
variables before moving to the phase II. The samples used in training and testing in these
three phases were randornly selected in order to eliminate bias. (See Chapter 5)
Multiple linear regression analysis (MLRA) was used to develop MLRA models for the
purpose of providing cornparisons to the ANN models. The results fiom the comparison
were then discussed. (See Chapter 5)
The research process is illustrated in Figure 1.1.
1.7. Principal Achievements
The main achievements of this research were:
Identified eighteen critical risk factors of project cost and time variations.
Concluded that risk factors such as project type, location, complexity, design
completeness, nurnber of key organizations, and weather are some of the major causes
of project cost and time variations. These factors have a mutual impact on project
cost and time outcomes.
Demonstrated that ANN technology has promising potential as an application in
project risk analysis.
Shown that ANN technology, combined with stepwise regression, is superior to stand
alone ANN or stand alone multiple linear regression analysis.
Suggested development procedures for ANN models based on this study and the
results fiom the research.
Created intelligent predictive models as a basis for developing decision support
systems in the future. This will help project managers develop project contingency
plans, screen alternatives and options and make better decisions.
Research Proposal
Perform Literature rn
Review h Establish Research Advisory Cornmittee
I Explore Significant Risk Factors 1
1 Design Survey Questionnaire 1
Test Questionnaire ?l
1 Analyze Raw Data 1
Prepare for Training 1 Develop Statistical Mode1 1 1 (Multiple Linear Regression) 1
Result Andysis r - l 1 Analyze Result 1
I Compare ANN and Statistical Model I
1 CONCLUSION (
Figure 1 .1 Research Process Flow Chart
1.8. Guide fo the Thesr's
Chapter two presents a background on project risks, risk analysis and relevant
techniques. The investigation of current usage and benefits of nsk analysis in industry is
presented. Existing nsk analysis techniques are reviewed and their strengths and
weaknesses are discussed. Finally, the data required for performing risk analysis is
discussed.
Chapter three descnbes ANN technology and provides a histoncal background of its
development. An ANN model is reviewed. Cornparisons between ANN vs. expert
system and between ANN vs. statistic analysis are presented. Current applications of
ANN technology in business and management and potentid applications of this
technology are also discussed.
Chapter four presents the detailed research methodology . The research method
undertaken includes an industry survey to gather data fiom completed projects in
industry. The details of the ANN model and learning algorithm and relevant issues are
then presented.
Chapter W e presents the development of the ANN models in three stages: preliminary
phase, phase I and phase II. The objectives of each of the three phases are presented in
detail. Two standard (ANN and multiple linear regression) methods and one hybrid
13
(ANN with multiple linear regression) approach are proposed and used to develop the
models to predict project cost and tirne variations. ANN models were aiso developed
using grouped data such as pipeline projects and refinery projects.
Chapter six provides conclusions to the research and recommendations for future
researchers and industry practitioners. Finally, fbrther research and contributions of this
study to the body of knowledge are discussed.
CHAPTER 2 REVIEW OF PROJECT RlSK ANALYSIS AND TECHNIQUES
2.f. Risk
Several definitions of risk appear in the literature. Papageorge [1988] defines risk as "any
exposure to a loss or damage". Doherty [1985] defines risk as "the lack of predictability
of outcornes". Cooper and Chapman [1987] defme risk as "exposure to the possibility of
economic or financial loss or gain, physical damage or injury, or delay, as a consequence
of the uncertainty associated with pursuing a particular course of action." No matter the
diEerences in words, the fùndamentals are the same. For the purpose of this thesis, nsk
refea to "the volatility of the outcorne" and is measured by "the deviation fiom expected
values." It can be either positive or negative.
The significance of risk sources varies thmugh project life cycles. Sources of risk often
cited in the literature and identified by industry practitioners include environmental,
regulatory, political, financial, scope, engineering, technology, complexity, changes,
project management skills and experience, contract and weather. These are ofien
considered critical risk factors in the early phase of a project and contribute signincantly
to project cost and t h e variances.
2.2. Risk Analysis
Cooper and Chapman [1987] state in their book, Risk Anuiysis for Large Projects, that
risk analysis cm involve a nurnber of approaches to dealing with the problems created by
uncertainty, including the identification, evaluation, control and management of risk.
Project risk analysis generally involves the assessment of the overall effects of these
factors, either implicitly or explicitly. This is frequently overlooked by owners who
evaluate nsks separately and qualitatively. The need for overall quantitative risk
assessment has increased with time largely due to global cornpetition, increases in
technological complexity, options and innovations, public involvement and regulatory
change.
Risk anaiysis may be appropriate in many circumstances throughout the life cycle of a
project, especially in the early stage in which uncertainty is significant and is thereby a
major factor. In the preliminary appraisal of a proposed project, a decision may have to
be made, often based on the owner's minimal amount of available information, to discard
the project, to postpone it, or to proceed with more detailed feasibility studies. For
example, a decision may be required to determine whether the project wilI be profitable.
This is determined by calculating the rate of r e m using the best estimates of capital
requirements and cash flows generated by the project If the resulting rate of return is
16
equd to or greater than the opportunity cost of capital, or the net present value is greater
than zero, the project should be undertaken.
Overall, there are many formai requirements for nsk analysis: economic viability,
financial feasibility, insurance purposes, project managers' accountability, contrachial
purposes, regulatory purposes, and communication purposes.
2.3. Cunent Usage and Beneflts of Risk Analysis in lndustry
An industry survey was carried out in late 1996 by the researcher. The objectives of the
survey were to investigate the current usage and benefits of nsk anaiysis techniques and
to gather state-of-the-art information on risk analysis practices in the oil and gas industry,
mainly in the Calgary area.
A formai questionnaire (see Appendix 1) was used and sent to fifty-five individuals, of
whom thirty-two responded. The response rate was 58%. Al1 the responses excepting
one fiom Montreal were used in data analysis since the study was focused in the Calgary
area.
Twenty-six respondents (84%) were fiom the oil and gas industry sector. On average the
thirty-one respondents had 18.4 years of project management experience. These
17
respondents' roles in project management included being project leaders, project
managers, project estimators, project engineers, project control managers and engineers
and project economists.
The survey results (see Appendix I for details) showed that thcre was a very strong
consensus of opinion (100% of respondents) that risk analysis must be used to assess risk
impacts on major projects (over $20 Million). Sixty-six percent of respondents stated
that risk anaiysis should be performed on projects with a value fiom $1 million to $20
million. Some of the respondents commented that conducting risk analysis was
dependent upon not only the size of a project but on the number of uncertainties
identified in a project.
The survey results dso showed that the selection of risk analysis technique was highly
dependent upon the nature of the problem. For instance, checklists and Monte Car10
simulation were the two techniques most fiequently used to assess project cost risks.
Checklists and CPMRERT were perceived as the two most useful techniques in the
assesment of risk in project scheduling.
Simister [1994] identified eight benefits of perfomiing risk analysis in his study.
Simister's eight items were incorporated in the m e y conducted by the researcher to
study the raak order of these items and to compare these results with Simister's hdings.
18
Participants were asked to identiQ what they thought were the most beneficial reasons for
conducting risk analyses. The eight benefits were ranked fiom the s w e y in the
following order:
1. Gives an increased understanding of the risks in a project
2. Allows the assessment of contingencies that actually reflect the risks
3. Allows the formulation of more realistic plans in terms of cost estimates and
timescales
4. Facilitates greater, but more rational, risk taking, thus increasing the benefits that can
be gained fiom taking risks
5. Leads to the use of the most suitable form of procurement/contract
6. Builds up statistical information about historical nsks that help mode1 future project
risks
7. Helps distinguish between good luck/good management and bad luckhad
management
8. Identifies the party best able to handle risks
There was a consistency between Simister's research and this study. The fhst three items
were considered the most important reasons in both studies. However, the rank order in
his hdings was slightly different compared to this study. The rank order in Simister's
study was #3, #1, #2, #4, #8, #5, #6, and #7. His study was conducted in the UK and the
19
industries he studied included defense, telecommunication, system-based information
technology and management consulting. Different countries and different industries may
cause the findings to be slightly different between the two studies.
In addition to the eight benefits listed above, participants to this s w e y identified the
following additional benefits to performing risk analysis of a project.
1. Assists in making the right decisions
2. Identifies golno go points and critical issues
3. 1s a part of the management of change process and assesses risk in order to set
priorities
4. Identifies critical activities that require risk management
5. Helps in the bidding process
6. Helps to determine the scope of projects during the conceptual phase
7. Assists in the selection of contractor(s)
2.4. Project Estimate and Contingency Allowance
Order of magnitude estimates are d y prepared in the early phases of a project when
project information is sparse and uncertainty is high. These estimates are often used to
assess the economic viability of projects or to compare dii3erent alternatives.
20
Project estimating should always deal with a range of outcomes. To cover uncertainties
and risks, practitioners usually build contingencies into the estimates. A large
contingency may make a proposed project unattractive. A low contingency may produce
inadequate hding. Therefore, realistic estimates are important to ensure the accuracy
and usefulness of: (1) economic assessments and (2) project cost and schedule control
baselines. Thus, realistic estimates indirectly influence project overall success.
Single point estimates are still popular in industry. Unfortunately, single point estimates
are not realistic as they miss a key piece of information: the probability of success. To
quote an expected cost of a project at $100 million means potentially different things. It
could mean the likely cost is $100 million, or it may mean that the estimate represents
absolute certainty (100% probability of success) or it could be an absolute minimum
required budget (100% probability of being over budget). A solution to this 'hizy'
situation is to apply a scientific approach, such as Monte Carlo simulation, to calculate
the probability of success associated with the single point estirnate.
A scientific approach u d y involves assessing the project outcomes' probability density
function and accepting certain levels of nsk. This approach relates a probability to the
docation of a contingency and uses the engineea' estimate as the basis for this analysis.
Doherty [1985] suggests using a contingency-allocation mode1 that includes the
following six logical steps:
1. Organizing and analyzing the estimating parameters;
2. Computing the estimator's base estimate;
3. Assessing the level of risk;
4. Assigning the classes of risk;
5. Establishing the even-chance estimate and the design contingency allowance;
6. Establishing the probability of success, the management contingency, and the cost
target.
The first two steps are based on traditional estimating methods. Risk analysis techniques
are brought in at the third step to assess overall project risk and variances. The
probability of achieving cost and duration targets associated with each contingency can
then be identifïed using the remaining three steps.
2.5. Review of Risk Analysis Techniques
The most difficult aspect of risk management is risk quantification. There are many risk
analysis techniques available to the project manager to assess project cost and schedule
risks. Some are qualitative methods (e.g., checklists), while others are quantitative (e.g.,
Monte Car10 simulations, regression). Some techniques can be performed either
qualitatively or both qualitatively and quantitatively (e.g., influence diagrams and
decision tree analyses).
22
Different risk analysis techniques can be used in different phases of a project. For
example, qualitative techniques are generally employed in the early stages of a project
where few or no precise measures or numeric data are available. Some existing
techniques and their strengths and limitations will be discussed in the following
~ a r w w h s +
Checkiist based techniques are among the simplest and most commonly used in industry
either formally (pre-stnictured) or informally (non-stnictured). Checklists are lists of
items that can be questions or categories used to gather information from a group of
experts (e.g., a project management team) to determine the identity of potential nsks, the
probability of occurrences and the potentiai severity of financialhime losses. Based on
the information gathered, project managers know what significant nsks have a high
probability of occurrence or can trigger potentially severe losses and what the risk level
of the overall project is. A contingency plan can then be allocated to cover the project
nsks. Risk mitigation plans c m be made to avoid or reduce the probability and/or
potentiai severity of those losses from occurring.
The advantages of checklist based techniques are their simplicity and ease of use. The
process is easily understood by project teams who have domain expertise but not risk-
analysis training. The success of the process is heavily dependent upon experience
combined with intuition and personal perspectives to nsks.
23
Monte Carlo simulation is a well-known quantitative tool. This form of analysis
typically starts with an established equation. For example, it is common to perform a
Monte Car10 simulation on a project cost estimate. The estimate is usuaily a
mathematical formulation of the following form:
N Estimated Total Cost = C Ci
i=l
where Ci is the cost of the i cost component of a project and N the total number of
individual cost elements. Every cost component with a potential for variabilit. is
modeled as a randorn variable. Others are treated as constant costs (those cost items
believed not to be expected to have any variations). Statistical distributions for each of
the random variables m u t be established before perfomiing Monte Car10 simulations. In
Monte Carlo simulations, randorn numbers are generated to produce values for random
variables based on pre-established distributions. The values produced and the constant
cost figure are added up to compute a value for the total cost of the project. This
procedure is repeated hundreds of times and a distribution is obtained for the total project
cost. The distribution can then be used to estirnate the project cost associated with a
probability of complethg the project on budget.
Monte Carlo simulation c m also be applied in the nsk aadysis of project schedules to
evaluate activities of uncertain durations identiflied within the Critical Path Method
24
(CPM) schedule. User-specified distributions for each uncertain activity are essential to
obtain in order to perform the Monte Carlo simulation. The probability density
distribution of the total project duration is presented after sufficient iterations. The
critical activities are those that appeared on the critical path in the largest percentage of
iterations during the simulation.
The distribution function of each random variable is initially established and is a matter
of judgment. It is often complex and difficult to obtain real probability distributions that
accurately reflect underlying uncertainties.
Monte Carlo Andysis can provide project managea with a range of estirnates and a
probability for each outcorne. However, this technique has some weaknesses identified
by several authors [Touran and Wiser, 19921 [Pedwell, Liu and Hartman, 19961
[Diekmann, 1 9921 :
Inability to hande situations where no explicit mathematical model is appropnate.
Inability to model or quantify actual distributions and correlations between individual
components.
Inaccuracies created in circumstances where there are important variables which are
not included in the mathematical model.
25
These disadvantages result in the limited use of Monte Carlo simulation in very complex
and unsûuctured situations. Care is recomrnended when using this simulation technique.
A study by Pike [1991] noted that although many managers are familiar with Monte
Carlo nsk analysis, cornparatively few use it.
PERT (Program Evaluation and Review Technique) is a well known but often
misunderstood approach to project scheduling. PERT was developed in 1958 for the US
Navy's Polaris missile/submarine project. PERT is performed in a similar fashion to
CPM (the Critical Path Method) which starts by establishing logic diagrams of projects.
Instead of using deterrnined durations for uncertain activities in CPM, optimistic
durations (a), most likely durations (b), and pessixnistic durations (c) of each activity are
used in PERT. The duration of each uncertain activity is calculated using the following
formula:
Therefore, project duration can be calculated and critical paths can then be identifïed.
This approach is an easy to use tool that can be used in project scheduiing involving
activities with uncertain durations. There are, however, two major shortcomings
associated with this approach. One is that it does not take into account the interactions
26
between activities because it assumes the duration variation of each activity is
independent. The other is that the determination of the three durations (a, b, and c) of
each activity is heavily dependent upon experts' experience and personal judgments.
Annlytic Hierarchy Process (AHP) is a risk anaiysis technique that provides a flexible
and easily understood approach to andyzing project risi<. AHP is a multi-critena
decision-making method that allows objective or subjective factors to be considered in
project risk analysis. This approach initially formulates the hierarchical structure of the
decision problem. Then, the relative importance of each element at different levels has to
be determined by decision makers. The total risk likelihood is detennined by aggregating
the relative weights through the developed hierarchy.
The AHP approach was intmduced and applied to the construction risk assessrnent of the
Jarnuna Multipurpose Bridge by Mustafa and Al-Bahar [1991]. The AHP mode1 is
illustrated in Figure 2.1.
GOAL
FACTORS
SUBFACTORS
LEVEL OF RISK
Constnrcting a Bridge Project
Total Risk TotaI Risk Total Risk
Figure 2.1. The AHP Risk Assessrnent Mode1
The approach provides valuable support for contractors in the decision making process,
but there are several weaknesses associated with this technique. The outcome of the AHP
is highly dependent upon the hierarchy established by decision malcers and the relative
judgments made about the various elements of the problem. The AHP approach aiso
lacks rigor. The effectiveness of AHP relies heavily on experts' personal intuition and
experience. This can lead to the inclusion of biases and manipulation, especially when
expert opinions conflict.
The Influence Diagram Technique is a method for representing the relationships of
decisions and uncertainties in a decision problem. The Decision Tree is a simple form of
infhence diagram. Basicaily, an infiuence diagram is constructed with a series of nodes
that are comected by lines (arcs). The nodes represent the uncertain variables in the
problem and the h e s represent the connections (influences) that exist between the
variables. By using influence diagramming, decision malcers seek to explain and
28
understand the complex causal relationships between decisions and outcornes. Influence
relationships (i.e. risk dependencies) are usually quantified using conditional
probabilities.
An influence diagram must be solved by the propagation of the influence of input
variables after an influence diagram has been constmcted. An example of influence
diagrarnrning with conditional probabilities is presented in Figure 2.2.
Material Unavailability a
P(W 1 IA 1): Probability S=S I given that A=A I
Adverse Weather Schedule Delay
(9
S2: Schedule delayed
M 1 : Material unavailable
M2: Materiai available
AI A2 A3 A 1 : Bad Weather: temperature <-20°C
Figure 2.2. An Example of Muence Diagramming with Three Variables
A2: Moderate Weather: 1 SOC >temperattue >-20°C
A3: Good Weather: temperature> 15OC
S 1: Schedule not delayed
SI
S2
0.7
0.3
0.2
0.8
1.0
0.0
29
The influence diagram technique is a common tool in the developrnent of expert systems.
The use of conditional probabilities to propagate influences is subject to one serious
shortcoming that depends on the topology of the influence diagram. It will become more
complex and subjective when a variable has more than one predecessor variable. The use
of influence diagram approach may be limited when the numbers of variables is so large
that the identification of dependent relationships between them becomes impossible.
Fuzy-Set Theory provides an unique approach to deal with decision problems described
with fupiness, vagueness and imprecision. Fuzzy sets are an attempt to capture the
richness of linguistic descriptions in a mathematical function Piekmann, 19921. This
approach is used
In risk analysis,
2.1).
to mesure qualitative subjects such as humanistic and societal factors.
risk interdependencies are measured using linguistic variables (Table
Table 2.1. Linguistic Variables for Determining Severity of Weather Risk
Linguistic Variables Definition
Low Less than 5% activities afTected by weather
Moderate 5% - 15% activities affected by weather
larger than 15% activities affected by weather
30
F u z y set fiinctions are then used to descnbe the linguistic variables quantitatively. For
example, low, moderate and high are defined as the foilowing funy set functions with
degrees of membenhip chosen so as to produce rectangular fuzzy sets.
Low = [O1 1 .O, 1 ~0.9,2~0.8,3~0.7,4~0.6,5~0.5,6~0.4,7~0.3,80.2,90.1,10~0.0]
Moderates [0~0.0,1~0.2,2~0.4,3~0.6,4~0.8,5~ 1 .0,6~0.8,710.6,8~0.4,9~0.2, 1010.01
High = [0~0.0,1~0.1,2~0.2,3~0.3,4~0.4,5~,6~.6,7~0.7,8~0.8,90.9,10~ 1 .O]
Then, fuzzy sets are manipuiated using fuPy algebra to propagate impacts of initial nsk
conditions throughout the influence diagram. The results then are defupified and
converted to linguistic terms. Figure 2.3 illustrates the steps of how fuPy-set theory is
applied.
1 IdentiQ the problem using influence diagrams 1
1 Assign linguistic variables I r I
Convert linguistic variables to fuzzy sets
Defutzify results 77 Convert fuzy sets to linguistic variables
1 Solutions to the problem I Figure 2.3. Steps of Applying Fuzy Set Theory
3 1
By using linguistic variables, fuzzy set theory can overcome the problem of complex
conditional probabilities used in influence diagrams. This modification allows experts to
express relationships in more natural, linguistic tems which are easily undentood.
Linguistic tems are subjective and need to be clearly defined and consistently understood
by experts.
Some authors [Wirba, Tah and Howes 19961 have demonstrated an application of fuay-
set theory in risk analysis in construction management. Linguistic variables through
funy sets were used to assess the likelihood of occurrence of each risk. The
effectiveness of the fuPy set approach to risk analysis was highly dependent upon the
d e f ~ t i o n s of risk dependencies and fiipv sets.
Regression AnaIysis is a generai statistical technique cornmonly used to solve important
research problems. Its uses range fiom the most general problems to the most specific.
This technique analyzes the relationships between a single dependent variable and severai
independent variables. This method relies on historic data and seeks to obtain patterns or
trends fkom these data. These trends are used to forecast future outcornes. An example
of a linear-multiple regression mode1 for a cost ovemm with two independent variables is
expressed as follows.
Actual Cost/Estimated Cost = a + bXi+cXi+dXj
Where: a is a constant
b,c,d is the coefficient of variable XI, X2, and X1, respectively
XI is the value of project type
Xz is the value of project complexity
X3 is an interaction variable between Xi and Xz
Merrow and Yarossi [1990] applied multiple linear regression analysis in the
development of a project evaiuation system which included a database and many
mathematical models. The system involved the assessrnent of cost, schedule,
performance, safety and market factors which posed risks to project success. Models
were built based on historic project data.
Federle and Pigneri [1993] introduced and developed a predictive model of project cost
ovemuis for the Iowa Department of Transportation (IDOT). The mode1 was built to
predict the amount of cost under/overrun based on several cost factors such as project
location, project type, project duration, etc. Multiple linear regression analysis was
applied aud seventy-aine projects were used to develop the model. The model helped
DOT engineers predict the potential for cost o v e m . The limitations of the model
were that it assumed observations were independent and it neglected possible interactions
between the cost factors.
33
Regression analysis is subject to a number of limitations. The use of historic data and
variables may not always be appropriate. The variable coefficients are obtained by
running the complete set of data. To incorporate new data, the complete set must be
reanalyzed. The mode1 mut also be specified in advance. If non-linear regression is
used, determinhg the exact nature of the non-linearity rnay be a burdensome task.
In summary, there are a numbers of risk assessment techniques in use today including
Monte Carlo simulation, analytic hierarchy process, fupy theory, etc. While some
positive results are available, they al1 suffer fiom one or more of the following
limitations:
assume uncertain variables independent
expert opinion and judgement
simplified models
simplified distributions of nsk variables
2.6. Data Required For Perfonning Risk Analysis
Numerou risk analysis techniques are available for the quantitative analysis of project
risk, but without quality data they are wortidess. Risk analysis software packages will
experience "garbage in - garbage out" if there is no quality data fed into them. Data may
34
corne from a variety of sources, representing the expenence of the project team, the
organization and the outside world [Bowers 19941:
Corporate: knowledge gained in past projects is dispersed throughout the
organization. Information may be stored as personal mernories, diverse reports
and databases that compare project plans and outcomes. Any organization is
encouraged to set up information databases to store past projects' data in a
consistent manner that may be valuable for future projects.
Project tearn: specific project experience is possessed by individuais within
project tearns. Such knowledge is probably relevant, but quite possibly limited
and biased.
Extemai: usefbl data may be obtained fiom other relevant organizations and
data warehouses,
Both academics and industrial practitioners have paid more and more attention to the
applications of quantitative risk analyses. Being aware of the strengths and weaknesses
of various approaches of risk analysis is necessary. Experts' expenence, personal insight
and judgment are heavily utilwd in most of the risk analysis models discussed above. A
recommendation can be made that the development of continuously updated knowledge
databases (project information, personal insight and experience, and organi;rational
information) is essentiai for an effective risk analysis.
CHAPTER 3 ARTlFlClAL NEURAL NETWORK TECHNOLOGY (ANN) AND APPLICATIONS
3.1. Introduction fo Aldificial Neurrl Nehvork (AIVN)
Artificial neural networks (ANN) are one of the fastest growing and most innovative
areas of computing. Neural networks represent an attempt to simulate hurnan thinking
processes through massively parallel, highly-interconnected processing systems.
Artificial neural network technology has been developing for several decades but has
ody found solid applications in the last decade. In recent years, ANN has been moving
fiom research laboratories into the business world. ANN applications are currently being
applied in many disciplines ranging from engineering to business management, especially
in fmancial and banking management.
3.2. Adficial Neurel Network Technology
3.2.1. Historical Background of Artificial Neural Network
Despite the curent attention of both researchers and practitioners, d c i a l neural
systems are not a new concept. According to Eberhart and Dobbins [1990], the history of
ANN can be divided into four periods of tirne: initial penod (1890-1969); depression
36
period (1 969- 1982); recovery period (1 982- 1986); heightened interest period (1 986-
present).
In 1943, McCulloch and Pitts suggested that a network of simple binary neurons could
perform highly cornplex cornputations. The neuron mode1 could perform logical
processing but people did not understand how information was stored in the model or
how intelligent behaviours were learned.
In 1949, Donaia O. Hebb postulated that information was stored in the connections
between the neurons, and that "leaming" consisted of modiwng these connections and
altering the excitory and inhibitor effects of the various inputs. Today, the major leamhg
paradigms for ANN are based on modifications of Hebb's original concepts.
In 1958, Frank Rosenblatt made a major contribution to neural network research with the
development of the perceptron which was the first real artificiai neural network. The
perceptron provided a simple model permitting extensive mathematical analysis of neural
networks and was simulated on a digital computer iBM704.
In 1960, Bernard Widrow and Marcian Hoff developed the Widrow-Hoff algorithm
which improved the speed of leaming and the accuracy of results. The algorithm was
widely used in back-propagation and other signal processing systems.
37
In 1969, MaMn Minsky and Seymour Pappert conducted an in-depth mathematical
anaiysis of the perceptron. Using simple examples, they showed that only a few
functions are guaranteed to be learned by the perceptron. In the case of the well-known
exclusive or (XOR) function, they showed that the function could not be learned by a
two-layer network. MaMn and Pappert's work had a devastating effect on neural
network research.
In 1982, John J. Hopfield introduced entirely connected networks called "Hopfield"
networks. He was one of the most important persons in the history of ANN development.
He re-triggered ANN researchers' interests after his several papers on the applications of
ANN were published. His work, and that of others, laid the foundation for significant
advances in neural network theory that has overcome the objections presented by Minsky
and Pappert.
3.2.2. Artificial Neural Network Mode1
An artificial neural network's paradigm allows it to mimic the fùnctions of a hurnan
brain. Information is distnbuted and processed in parallel within an ANN. ANNs
exhibit certain feahires such as the ability to lem, recognize trends, and simulate human
thinking processes. Today ANNs can be trained to solve problems that are difncult for
conventionai cornputers or even for h u . beings.
38
An artificial neural network consists of many simple-processing elements called neurons.
Each neuron has a small amount of local memory and some elementary computing
power. Neurons are c o ~ e c t e d to each other. A neuron receives input fiom other neurons
on incoming pathways and has only one output which can be directed to many other
neurons. In Figure 3.1 a single neuron with r inputs is show. The individuai inputs Po),
weighted by elements W(1 j) of the matrix W, are summed to form the weighted inputs to
the transfer fùnction F. The neuron has a bias b and an output A.
Input Neuron
7 0 k
Figure 3.1. A Multiple Input Neuron Mode1
Neurons communicate only through these input/output pathways. A neuron cannot
access the memory of other neurons. Each n e m n has a transfer h c t i o n which is used to
compute the output fiom the inputs. Different neurons in a network can have different
transfer bctions .
A neural network typicaily consists of six primary components: neurons, connections
be~reen neurons, weights, transfer fiinctions, leaming d e s , and the overail
transformation process. These components determine the artificial neural network
paradip. The details on these topics are discussed in Chapter 4.
There are two basic types of connections: feed-forward in which neurons in one layer are
connected only to those in higher layea; feed-back in which any neuron can be connected
to any other.
There are three possible learning methods: supervised, unsupervised, and reinforcement
learning. With supetvised leamllig, the desired output for a set of training set is provided
to the network; thus it lems by example. UnsupeMsed learning is conducted when there
is no desired output provided to the network. The network defines its own output set
based on relationships derived kom the training set and learns by self-organization.
Reinforcement leanillig is a hybnd method, the network is given a scalar evaluation
signal instead of behg told the desired output, and evaluation can be made intemiinently
instead of with every nainllig input set.
Many different artîficial neural network models have been proposed that address many
distinct and diverse problem domains [Wilson 19901. One such usefùl neural network
model, especiaily for business and management applications, is the multi-layer, feed-
forward model - backpropagation. The model typicaily has one input layer, several
hidden layers and one output layer. Neurons on one layer are comected only to those in
40
higher layers and the output of one neuron on one layer becomes the input of the neurons
on the next higher layer. A neural network mode1 with three layers is shown in Figure
3.2.
1 Xr - - - - - Input Amy
Hidden Layers
Output Layer
Figure 3.2. Neural Network Mode1 with Three Layers
The back-propagation algorithm attempts to mlnimize the sum of the squares of errors at
the output layer during the training process. A training set is comprised of pairs of input
values and the desired output values (used for a s u p e ~ s e d learning process). A training
set is presented to the neural network and the activations are fed forward through the
network, resulting in output at the output layer. This output provided by the network is
compared to the desired output for the particular input set. Network weights are adjusted
such that the ciifference between the network output and the desired output is minimhed.
Adjustments due to the output error are propagated backward through the network,
starthg at the output layer and moving back toward the input layer. The procedure is
41
repeated over the training set until the network converges and produces the desired
responses within certain error cnteria.
ANN can be represented mathematicdly in terms of vector and matrix algebraic systems,
with the input and output values described in tems of vectors, the topology in terms of a
comectivity matrix consisting of the weights of various connections, and the learning
d e as a differential equation relating the output-target value pair, and the weights.
Training the network consists of solving this differential equation to determine the next
set of weights to be used to generate an output. Successful ANN paradigrns allow the
network to converge on stable solutions.
The training of a neural network takes place in the following conditions: a significantiy
sized training set is available; a multi-layered network is initialized with random
interconnection weights. The size of the training set and the structure of the network are
important when training a neural network.
3.3. Compdson between ANN's and Ofher Techniques
3 . 3 Artifhial Neural Network vs. Expert Systems
Neural networks offer a novel approach to the decision making problems where there is
no information available regarding the assumption of data distributions or relationships in
the categorization dilemrna. Thus, problem domains such as unstructured, fuzy, or
nonlinear are ones where neural networks may represent substantial information
processing improvement.
Expert systems represent one of the most important developments in information
technology. Based on Artificial Intelligence (Ai) concepts, expert systems attempt to
incorporate the reasoning process and knowledge of experts into cornputer prograrns.
Expert systems were first developed in the 1960s and became commercially available in
the early 1980s. Commercial applications can be found in many industries that include
aerospace, rnilitary, banking and financing, manufacturing, retail, personnel management,
marketing planning, etc.
In contrat to expert systems, neural networks represent an innovative technology that
uses parailel information processing. The parallel architecture of neural networks makes
them particularly adept at analyzing problems with many variables by simultaneously
considering numerous factors. The pattern recognîtion and prediction capabilities of
43
neural networks make them suitable for a number of business and management
applications.
Expert systems and neural networks differ in a nurnber of ways (refer to Table 3.1). One
key ciifference is their foundation. Expert systems are based on logical sequential
processing, and neural networks use parallel processing to attempt to simulate
intelligence. In expert systems leaming usuaiiy takes place outside the system.
Knowledge is obtained outside and coded into the knowledge base. In contrast,
knowledge in neural networks is stored as the weights of the connections between the
neurons. The leaming process is interna1 to the networks and can be dynamic. The logic
for processing knowledge in the two systems is different. Expert systems use deduction,
and neural networks use induction.
Table 3.1. Features of Expert Systems and Neural Networks
Expert Systems Neural Networks
Macro scope Micro scope
Sequential processing
Learning takes place outside systems
Deductive
System buiit through knowledge extraction
Mathematical logic origin
Exact rnatching
People Oriented
Knowledge is explicit
Parallel Processing
Learning takes place within systems
Inductive
System B d t through training
Statisticai and stochastic ongin
Approximate matching
Data oriented
Knowledge is implicit
44
The underlying theones of the two technologies differ. Expert systems are based on
mathematicai logic "IF-THEN", following an objectsriented approach. Neural networks
are statistical and stochastic in origin. In an expert system the user cm query the system
to determine why and how output was derived. In contrast, the knowledge base in a
newal network remains a "black-box" to users.
Table 3.2 shows the strengths and weaknesses of expert systems and neural networks.
Table 3.2. Strengths and Weaknesses of Expert Systems(ES) and Neural Networks(ANN)
ES ANN
P
Exphnation No Explmation Capaciîy Capacity
Many turnkey Most ANN m u t be shells are customized per available application
Strong user Weak user interface interface
Software well Hardware and developed software are in
development stages.
Easy to validate Difficult to validate
Requires an articulate expert to develop ES
Requires a significant number of exarnples
Average developrnent time is 12 to 18 months
Leaming is static and extemal
Data should be complete and error fiee
Knowledge engineering is difficult and time- consuming.
Development tirne is as littie as a few weeks or months
Leaming is dynamic and intemal
Data can be incomplete, error ridden and noisy.
Knowledge engineering is data driven and simple.
45
Neural networks have many benefits in terms of knowledge acquisition compared to
expert systerns. In many unstmctured decision environments, particularly those that
involve classification, associative memory, or clustering, neural networks offer distinct
advantages over expert systems. One of the most significant strengths of neural neh~orks
is their fast computation and the possibility to convert a neural network to an electronic
chip using VLSI technology. Expert systems have some advantages that neural networks
do not have. They have user-fiiendly interfaces that provide explmations to the choices
made by the system and have an interactive, dynamic, and visual problem-solving
capability [Osyk and Vijayaraman 1 9951.
Several authoa have attempted the integration of expert systems and neural networks.
Expert systems can be employed to train neural networks, control information flow
through several neural networks, and anaiyze the responses provided by the neural
networks [Wong and Monaco 19951 . The greatest potential in AI technology may lie in
combining neural network hardware with expert system software [Duggai and Popovich
199249931. integration of fuzy logic and neural networks is another potential to deal
with unstmctured, fuzzy and large information problems [Bataheh 1 995- 1 9961.
3.3.2. Neural Networks vr. Conventional Statistic Analysis
ANNs are distributed, parallel information-processing systems that exhibit certain
features such as the ability to learn, recognize trends, and capture patterns. ANN's ability
to build the relationships between input data and output data can be used to tackle
problems that have been conventionally handled by statistical methods such as regression
anal y sis, discriminant analy sis and cluster anal y sis. Cornparison of neural networks with
more traditional statistical techniques has been the focus of many recent studies. Some of
the main differences between regression and neural networks are addressed by authors
[Bansal, Kaufian and Weitz 19931. These are as following:
Neural networks consistently improve during training. Regression
techniques, on the other hand, process al1 training data simultaneously, before
using new data;
in theory, neural networks may be more robust than nonlinear classification
models;
Regression equations repuire mode1 specification in advance. In nonlinear
regression, specifjhg the exact nature of nonlinearity may be a burdensome
task. Neural networks avoid mode1 specification in the regression sense but
require specification of neural network architecture.
47
Neural networks perform relatively well with missing or incomplete data, whereas
regression does not. Neural networks also typically have been shown to produce more
accurate predictions with good-quality data than regression models [Dutta and Hekhar
19881. De Groot and Wurts [1991] concluded that neural networks are the best
approach when data exhibit non-linear characteristics. However, neural networks are not
able to explain how they solve problems due to their "black-box" nature; no decision
d e s are generated for a decision maker's reference and inspection.
3.4. Issues Relafed fo Neural Network Taining
Numerous studies have unquestionably shown the utility of ANN models. However,
some issues must be addressed when training an ANN. These issues include:
The learning rate is a parameter that affects how neural connection weights are
updated. If the learning rate is large, it may cause rapid error correction in the
network, but it may potentially lead the aaining to non-convergent solutions. A
lower learning rate may be appropriate in allowing the nework to gently
converge, but it may cause the network to require more iterations through the
training set.
Training tolerance values represent the aliowable variation between the acnial
output set and the desued output of the training set.
48
The stopping cnteria for training a neural network can be that the network
reaches either its traùillig tolerance or its maximum number of iterations.
Composition of the training set is an important consideration in neural network
training.
The size of the training set will have impacts on network training. It is intuitive
that the larger a training set, the more rich that set is in describing the problem
domain and might therefore result in a more accurate network than with a
smaller training set.
The parameterization of a neural network is an important issue. It has been
postdated that the configuration of the network may affect the accuracy and the
generalizability of the trained network. Network pararneterization includes the
number of input neurons, the number of output neurons, the number of hidden
layers and neurons, and the transfer function. The numbers of input and output
neurons are determined by the nature of the problem to be solved. The number
of hidden layers and neurons on each of them are established during the training
process for efficient performance.
There are few d e s or guidelines to denve a network configuration for a specific
problem. The appropriate number of hidden neurons and layers remallis an unsolved
research question.
3.5. Applications of Arüficial Neural Network in Business and M8nagement
The field of neural networks has a five-decade history but has found solid applications
only in the 1st decade, and this field is still developing rapidly. Neural network
technology has been applied in a wide range of areas, such as image recognition, signal
processing, control systems, forecasting, text retrieval, and optimization.
In recent years, neural network technology has been moving fiom research laboratories
into the business world. A number of practical applications of neural networks have
emerged in a variety of business and management fùnctions fiom banking and finance to
quality control and natural resource exploration.
Sorne figures can give an indication of the intense investment of neural networks. In
1986 the U.S. govemment was advised to commit over $400 million for research over an
eight-year period. in 1988 the private sector invested an estimated $20 million in the
purchase of neural network systems. About 80 percent of the Fortune 500 companies
have an investment in neural networks [Johnston 199 11.
Although the fastest growing sector for neural network application development is
defense, in private industry, the hancial services industry was one of the earliest
adopters of neural network technology. As we know, many highly stnictured decision
problems c m be solved effectively by conventional digital computer systems. However,
50
most top-level decision making problems faced by financial managers are highly
unstnictured, very complex, and not easily adapted to conventionai approaches of
cornputer-aided analysis and decision support. Neural networks offer distinct advantages
to help solve these problerns.
In recent years, academic researchers have explored the use of neurai network technology
for bankniptcy prediction [Koster, Sondak and Bourbia 1 990- 199 11, bank failure
predictions [Tarn and Kiang 19921, optimum markup estimation Noselhi, Hegazy and
Fazio 19911, cost engineering pck im 19931, portfolio selection [Suh and LaBarre 19951,
strategic management [Slicher, Vakalis and Singh 19951, and environmental engineering
[Basheer and Najjar, 19961. But more companies are exploring the commercial use of
neural network technology for tasks such as credit card h u d detection, signature
verification, and evaluating the financial audit process.
AVCO Financial Services, a division of Teatron, Inc., has successfidly implemented a
neural network for a n a l m g loan applications. Banc Tec, Inc. plans to introduce a neural
system to read handwritten numbers on checks, now a human chore. Adaptive System
Inc. has developed neural networks for assessing mortgage applications. Chase
Manhattan Bank, the second largest issuer of bank credit cards, announced the installation
of a neural network to detect credit card hud. Citicorp's Quotron Systems is using
neural network software c d e d Braincel to predict stock-index movements five minutes
51
ahead of t h e . Neural Systems Inc. makes use of a supervised network to mimic the
recommendations of money managers on the optimal allocation of assets among Treasury
instruments. Ward Systems Group, hc . created an example showing how one might set
up an ANN application to predict stock market behavior. Companies such as Ford,
Morgan Stanley, AT&T, and Raytheon are already exploring their potential in
applications as diverse as faultfinding, equities dealing, and sonar detection systerns. In
1994, the Canadian Imperia1 Bank of Commerce (CIBC) replaced its index-based
Leading Indictors with a neural network-based system [Tal and Nazareth, 19951. These
systems are frequently used for tracking general economic direction and their
performance to date has been very encouraging.
3.6. Potential Applications of Neural Network in Pmject Risk Analysis
The early decision making system for project definition and selection is quite a complex
process due to risk and uncertainty. The system can be defhed as a nonlinear and
unsû-uctured probiem. Risks, project decisions and project outcornes are interrelated.
The relationships between process variables cannot be explicitly represented in a
mathematical model.
In this study, project decisions c m inciude, for instance, fast-tracking considerations and
contractuai strategies. Project risks can be, for example, weather and technology factors.
Project outcornes refer to cost and t h e variances. These are correlated with each other
and compose a complex and uncertain system during the early stage of a project.
Project and risk anaiysis techniques are currently used in project cost and time estimating
and evaluation of risk impacts. The use of conventional risk analysis techniques is
usually based on well-defined and stnictured mathematical models. These techniques are
not suitable to solve complex and unstructured problems.
Mathematical fiee methods, such as aaificiai neural networks, provide a powerful and
potentiaily robust means to assess uncertainties through learning and capturing general
patterns in available data. ANN are usually used where problems cannot be readily
represented by explicit analytic models, and where the process of reasoning to obtain a
result may not be logical, and where signifiant data exists. ANN technology promises to
provide an innovative approach to project nsk analysis, especially in the early stage of a
project.
CHAPTER 4 RESEARCH METHODOLOGY
4.0. Literatum Search
A broad study in the areas of project management, risk analysis and management,
information systems and technology, artificial neural network technology and its
applications was conducted. This began in early 1996 and continued throughout the
research period to ensure that new information, research ideas, and applications were
captured.
The literature search included books, technical and research reports, newsletten, theses
and dissertations, proceedings and journal articles. About thirty journals and proceedings
were reviewed for relevant information. Hundreds of related papers and articles were
reviewed and backed up for references and M e r research.
Sources for references included:
Mackimmie Library, the University of Calgary (stacks, CD-Rom)
The Scurfield Management Library, the University of Calgary
I The Internet
Relevant conferences
The City of Calgary Main Library, Calgary
Foster Research Center, Calgary
Project Management Body of Knowledge, PMI Standards Cornmittee, PMI USA,
1996
Research Reports by the Construction Industry Institute, The University of Texas at
Austin
The review concentrated on sources fiom the 1990s and the late 1980s because ANN
applications in business and engineering only began in the 1980s. Reference lists in
journal articles were scanned and relevant sources were located for further review.
Relevant idormation was noted for closer review and copied for backup. Al1 backup
information and articles were sorted into foldee fustly by subject area and secondly by
publication year. The research interest and cornrnon themes in the each area can be
identified year by year in this system.
4 . lndusfry Survey
4.1.0. Introduction
A large amount of completed project information was needed to fullill the objectives of
the research. An industry survey was designed for this purpose. Although there were
55
many industrial sectors of interest, the oil and gas industry was the focus of this study
because it is a major industry in the Calgary area. Tirne constraints on performing the
research were a secondary reason for limiting the scope to one local industry.
4.1.1. Research Advisory Cornmittee in Industry
M e r the research proposai was approved, a research advisory cornmittee comprising four
people was established to direct and guide the research and to provide support, especially
for the industry survey. Each cormnittee member had significant project management
experience and was a recognized leader in the oil and gas industry.
Each committee member committed his tirne, effort, and insight throughout the research.
Each member was updated with feedback in the fonn of research progress reports.
Suggestions from cornmittee members were considered and incorporated into the
research.
4.1.2. Ethics Approval on Human Studies
According to the univeaity policy regarding the ethics of human studies, conducting
research involving human subjects without formal ethics approvaVclearance breaches the
university's polîcy on Integity In Scholarly Activity. Prior to carrying out the research,
approval from the Committee on
form and the research proposal
56
the EWcs of Human Studies was obtained. A consent
were attached to the application form for review by
cornmittee members. Certification of Institutionai Ethics Review was issued after the
cornmittee examined and approved the research proposai. The approved consent fom is
presented in Appendk II.
The consent form was sent to each potential survey participant with the survey
questionnaire. Each participant was asked to read, agree with, and then sign the consent
forrn and retuni it to the researcher.
4.13. Expert Panel Intewiew
An expert panel interview was conducted to gather information and identiQ indu-
practices related to project risk analysis and management in a relatively short time frame.
The objectives of the expert panel survey were to:
Ensure indusüy interest and support of this research;
0 Observe curent practices with respect to risk identification, risk analysis and
management;
Idente the critical factors that signiticantly affect project cost and time variances.
57
Expert panel interviews were used to gather information that enhance and address the
gaps in knowledge in the research area between literature and professional practices.
An expert panel of eight people, who had rich and extensive experience in nsk and
project management, was selected from both academia and industry. These experts were
fiom owner organizations, enginee~g consulting companies, construction f m s and the
universities. An open-ended questionnaire was used. Critical factors that impact project
outcorne (cost and tirne) variances were identified by the expert panel and used to design
a draft survey questionnaire.
4.1.4. Design and Testing of Survey Questionnaire
A draft structured questionnaire form was designed by compiling the information and
knowledge fkoin the literature supplemented by the expert panel's knowledge, insight and
practices.
A bnef introduction of the purpose and objectives of the study was covered on the fk t
page of the questionnaire form. Confidentiality of survey data and approaches to the
management of the raw data were also addressed. In addition, instructions were provided
to potential participants on how to m e r the survey questions properly.
The survey questionnaire was compnsed of the following four sections:
Project details. In this section, participants were asked to provide the estimated cost
and actual cos4 estimated and actual duration, and the year of completion, of the
surveyed project.
Project critical factors that impact time and cost variances. Critical factors for
projects were identified as project type, location, complexity, design completeness,
level of scope definition, project management experience, contract type, project
priorities, number of regdatory permits required, level of technological innovation,
weather, cost spent on the fiont end planning and cost vent on detailed
engineering/design.
Top ten signifcant factors injluencing cost and time variances for the speczpc
project. This section asked participants to identi@ the top ten factors from section 2
above that had significant impacts on the specific project.
Additional information. This section asked participants to provide additional
information not included in the survey questionnaire form that they felt was critical to
the project. Participants' information, such as current position, name, mailing
address, and their interest in receiving a brief copy of the study results, was dso
reauested so the researcher could provide feedback to s w e y participants.
59
Language, phrashg and temiinology can pose sigaificant problems in a survey. Testing
is necessary to detect weaknesses in the questionnaire. Testing was undertaken through t
persona1 in te~ewing of colleagues, practitionen, some potentid s w e y participants, and
academics. The purpose of the testing was mainly to:
discover respondents' reactions to the questions;
check question interpretation;
use easily understood terminology;
check continuity and 80w.
A variety of valuable comrnents and suggestions fkom these tests were generated. Al1
comments and suggestions were taken into consideration and, where appropriate,
incorporated into the survey. The fmal version of the survey questionnaire is attached in
Appendix m.
4.1.5. Sample Design
The sample population was identifïed as oil and gas hdustry companies including owner
organi;rations and EPC consulting companies. The snowball sampling technique was
used for the following reasons:
60
The survey required that participants not only have sufiicient knowledge and
experience in project management but that they were also directly involved in the
projects surveyed.
Respondents were difficdt to identifjc
The major concem was to gather information fiorn as many as possible completed
projects within a designated time frame.
The snowball sampling design has found a niche in recent years in applications where
respondents are difficult to identify and are best located through referral networks
[Cooper and Emory, 19951. In the initial stage of snowball sampling, individuals are
selected and then used to locate others who posses similar characteristics and who, in
turn, identifi othen.
The snowball technique allows the researcher to staa with a list of referrals and then
identiQ more potential participants fiom names provided by the previous respondents.
After each interview, conducted over the phone or in person, each participant was asked
to provide names of other potential participants that the researcher could contact and
suwey.
4.1.6. Administration of the Survey
To emure a high response rate among participants, al1 surveys were sent out following
initial contact. Each potential survey participant was contacted individually by phone or
fax or email to get hislher commitment to participate in the research.
Surveys were sent out only to people who made a commitment to participate. Each
s w e y included a deadline that was approximately five months after the initiai contact. If
no response was received after one month, one or two follow-up telephone calls were
made to remind participants and to check whether they were having problems with the
sumey.
Many diEculties were encountered during the survey, even though most of participants
were willing and committed to participate in the swey. These difficulties were caused
by several reasons:
No hands-on data was readily available and participants needed to look back into the
specific completed project fiie(s);
Organhtion dowosizing;
Lose of key people;
People were too busy to be able to respond.
62
Completed sweys were mailed or faxed back to the researcher. AU survey data were
coded and entered into a spreadsheet for raw data analysis. No third party was allowed to
access the data file.
4.1.7. Raw Data Analysis
A simple descriptive analysis was carried out for gaining general information on the
overall collected projects. The standard deviations and the average values of cost
overruns and t h e overruns were calculated. Collected projects were categorized to
define the distribution of the project type in the oil and gas industry. Results fiom and
discussion on the descriptive analysis are in Chapter 5.
4.2. DEVELOPMENT OF ARTIFICIAL NEURAL NETWORK MOOELS
4.2.1. Introduction to ANN Mode1
The development of neural network models is highly problem-dependent. Developing an
ANN mode1 requires the determination of an ANN paradigm. This includes ANN
learning algorithm, the numbea of layers and neurons, transfer bction, etc. An ideal
paradigm must optimally describe the nature of the study's system and perform well
63
according to certain criteria. This results fkom a cornplex and the-consurning training
process. The following subsections address these related issues.
4.2.1.1. Artificial Neural Network Leaming Algorithm
The first step in the development of the ANN mode1 was to select a type of neural
network. In this study, a feed-forward network with the error back-propagation algorithm
was selected and used to develop predictive models of project cost and time risks. This
type of network wiih back-propagation algorithm provides a theoretically sound method
for the supervised training of the networks and requires a continuous transfer fùnction
wasserman and Schwartz 19871. The several reasons for choosing this type of learning
algo rithm included the following :
i It was able to train a multiple-Iayer network;
H It was capable of exploithg the regularities and exceptions in the training samples;
i It guaranteed stability of the network;
Figure 4.1 presents a sample of an ANN feed-forward network with three layers and ten
hidden nodes. Each of the nodes in the network is a neuron. Each of the links is called a
comection. As we can see, neurons in one layer are connected only to those in a higher
layer in the feed-forward type network.
Cost Time Variance Variance (%) (%)
Output Layer
Hidden Layer
Input Layer
Figure 4.1. An ANN Mode1 With Three Layers and Ten Hidden Nodes
Back-propagation algorithm was based on the error-correction l e h g rule and the
neural network was trained in a supervised manner in order to map the relationships
between the input samples and the output sampies. Basically, the error back-propagation
process consisted of two passes through the different layers of the network: a forward
pass and a backward p a s .
1. Forward Pass
In the forward process (Figure 4.2), an input vector (Pi) was introduced to the input nodes
of the network. Each connection between the input layer and the hidden Iayer had an
65
associated weight (IVji). The net signal (Ij) to an individual hidden node (j) was expressed
as the sum of al1 the connections between the input layer and that particular hidden node
plus a bias (Bj), as in equation (4.1).
Where R is the nurnber of nodes in the input layer.
The signal fiom the hidden layer was then processed with a tmnsfer function (F,) and the
output signal Oj was generated pnor to being sent to the output layer. The procedure was
performed according to equation (4.2).
Input Nodes
Hidden Node j
Oj Pi
Forward Pass
Figure 4.2. Signal-flow Graph: Two-Layer Network
66
Each connection between the hidden layer and the output layer had an associated weight
) The net signal (Pd to the output node (k) was the sum of al1 the connections
between the hidden layer nodes and the output node plus a bias (Br&, expressed as:
where Q is the number of nodes in the hidden layer.
The net signal utk) was again transferred to the final output value (0'3 with a &ansfer
function (F3, expressed as:
Finally, a set of outputs (0') was produced as the actual response from the network.
During the forward pass the weights of the network were al1 fïxed. The initial weights
and biases in the network were usually randomly genenited.
2. Backward Pass
During the backward pass the weights were ail adjusted in accordance with the error-
correction rule. The weights were adjusted so as to make the acnial response of the
network move closer to the desired response. The backward process is shown in Figure
4.3.
Bac kward Pass -1
Figure 4.3. Signal-flow Graph: Two-Layer Network
At the output layer, the net signal (0'3, which estimated outputs fiom the network, was
compared to the desired output (TJ to produce an error signal (E,) which was propagated
back through the network and expressed as:
Ek= Tk - O',
Between the output layer and the hidden layer, the correction A W M and applied to
Wkj and Brk, respectively, were defined as:
where q is the rate of ieaming; and 6'k is cailed local gradient, defined as:
where F', is the derivative of the transfer function F,.
Between the hidden layer and the input layer, the correction AWji and ABj applied to Wji
and Bj, respectively, were defined as:
where 6j is defined as:
where F', is the derivative of the transfer fiuiction F,; S is the number of output nodes.
3. Iteration
The computation through forward and backward passes by presenting a set of training
examples to the network is called an iteration. The set of training examples was fed
69
repeatedly to the network until the network converged to a stable solution and the sum of
the squared error (SSE) of the network computed over the entire training set was at a
minimum or acceptably small value, expressed as:
where N is the number of training examples; n is the nfh training example; s is
predefined error criterion, also called training tolerance.
4.2.1.2. Layers and Neuron Nodes
The second issue in the ANN modeling was to defme layea and the number of neuron
nodes on each layer. Three-layer neural networks were w d in this study. The first layer
was called the input layer which consisted of eighteen nodes. Each of the input nodes
represents an independent variable. The input variables are listed in Table 4.1 (See
Appendix IV for details). The M d layer was the output layer which consisted of two
nodes representing cost variances (%) and time variances (%).
Table 4.1. Input Variables
1 I n ~ u f Variables 1 Label 1 Type of Project 1
Location of project
Complexity
No's of key organization
safeîy I v9
Design completeness
% of cost spent in front end
% of cost spent in detailed engi.
QditY
Input Variables 1 Label 1
V5 V6
V7 V8
Sc hedule I I
Years of experience I VI5 I
Cost
No.s of permitslapprovals
Scope defïned
Previous experience
Type of contract 1 VI6 1
VI1
VI2 V13 V14
m
Weather 1 V18 1 Level of technology
The number of hidden layers was determined through the training process for the
effective performance of the model. Typicaily, the more neurons in the hidden layer the
more powerful the network. The typical result of not using enough neurons in hidden
layers is that no set of weights and biases such that the network can produce outputs that
reasonably close to the targets are produced. This eEect is called underfitting. Too many
neurons in the hidden layer cause speciaiization effects that miss key points and cause
errors [Koster, Sondak, and Bourbia 1990-199 11. It may lead to fitting (or memorizing)
the training sets too weii resulting in poor generalization capabilities. To overcome these
problems, a range of hidden neurons was used in the training process.
V17
Hecht-Nielsen [1989] and Caudill Cl9911 recommend that any continuous fiinction cm be
Unplemented with a muiti-layer network using 2n+l hidden nodes, where n represents the
number of input nodes. Fletcher and Goss [1993] stated that in practice, the number of
71
hidden nodes for optimal generaiization should be tested in a range fiom approximately
2*sqrt (n)+m to the value 2n+l, where m represents the number of output nodes and n
represents the nurnber of input nodes. In this study, the range of hidden nods was nom
2*sqrt (n)+m to 2n+l.
4.2.1.3. Transfer Functions
Each neuron had a transfer function that was used to compute the output signal fiom the
input signals. Three of the most commonly used transfer Functions for neurons being
trained with back-propagation are Linear, Tan-Sigmoid and Log-Sigrnoid.
1. Linear (Pureline)
The linear transfer functions shown in Figure 4.4.a and Figure 4.4.b have an output equal
to its input plus the bias. This kind of function is often used with neurons being trained
with the back-propagation.
Figure 4.4.a. Linear without Bias Figure 4.4.b. Linear with Bias
2. Tan-Sigmoid (Tansig)
Figure 4.5.a and Figure 4.5.b show the tan-sigmoid fûnctions with and without bias. The
tan-sigmoid function is a differentiable hct ion and is used to map a neuron input fiom
the interval (-a, +a) into the interval (-1, +l), i.e., the output is dways between +1 to -1.
Figure 4.5.a. Tan-Sigmoid without Bias Figure 4.5.b. Tan-Sigrnoid with Bias
3. LogSigmoid (Logsig)
The log-sigrnoid tramfer hctions shown in Figure 4.6.a and Figure 4.6.b squash the
input (which may have any value between plus and minus infini@) into the range of O to
1.
Figure 4.6.a. Log-Sigmoid without Bias Figure 4.6.b. Log-Sigmoid with Bias
The log-sigrnoid is a differentiable function that makes it suitable for neurons being
trained with back-propagation.
The transfer functions on the nght in each of the three cases shown above have a b i s ,
whereas those on the left do not. A bias can be a constant or allowed to change Iike the
weights with an appropriate learning d e .
The three transfer functions discussed above were used in the neuron network training
process. The possible combinations of these functions, such as using the linear fiinction
for ail nodes on the hidden layer and using logsigrnoid for al1 output nodes, were tested
in a training process in order to search out the best combination of those three transfer
fùnctions for this typical problem.
4.2.2. Artifhial Neural Network Training Procedures
Training an ANN is the task required to arrive at a unique set of weights that are capable
of correctly associating al1 example patterns, used in leamllig, with their desired output
patterns. The algorithm cycles through the training data { [P(n), D(n)]; n = 1,2, . .., N] as
follows, where D(n) is the desired output vector corresponding to the input vector P(n).
1 . Initialization. Start with a reasonable network ~ o ~ g u r a t i o n , and randomly set al1
weights and biases.
2. Presentations of Training Examples. Present the network with a set of training
examples. For each example, perform the following sequence of forward and
backward computations described under step 3 and step 4, respectively.
3. Forward Computation. Let a training example in the epoch be denoted by [P(n),
D(n)], with the input vector P(n) applied to the input nodes and the desired output
vector D(n) presented to the output nodes. Compute the network outputs vector O'
according to equation (4. l), (4.2), (4.3), and (4.4).
4. Backward Computation Compute 6's of the network by preceding backwards, layer
by layer, according to equation (4.8) for output nodes, and equation (4.1 1) for
hidden nodes-
5. Iteration Present examples one by one in the training set to the network and repeat
step 3 and step 4. Repeat the training processes many thousands of times until a
certain preset criterion to stop the leaming session is met One such critenon is to
75
consider the network to have adequately learned when the error between the output
produced by the network and desired output, accumulated in al1 leaming examples,
is less than a specified limit. The "stopping" point can also be determined by an
acceptable number of training iterations.
4.2.3. Issues Related to Am Training
4.2.3.1. Learning Rate, Momentum Constant and Training Tolerance
Multilayered nonlinear networks trained with back-propagation are sensitive to the
learning rate. The learning rate is a parameter that affects how neuron connection
weights are updated when back-propagating error toward the input layer. The greater the
learning rate, the more radical change an output error in the training process causes in the
connection weights. Thus, a large leaming rate rnay cause rapid error correction in the
network, but potentiaily the network solutions will keep jumping over the error minimum
without converging. A smaller learning rate may cause the network training to require
more iterations through the trainhg samples. So, a suitable leaming rate has to be
obtained by experimentally testing different Ieaming rates. In this study, a leaming rate
of 0.001 was gained after many experiments. It was believed that it was an appropriate
leankg rate to this specific problem and used through aIl of the training processes.
76
Nonlinear networks introduce a complication because they may have several local
minima. Ideally, the network should fmd the global minimum. But this cannot always be
guaranteed. A network may get "stuck" by rolling into a local minimum [Demuth and
Beale, 19951. The authors suggest a technique called 'momentun' in ANN training to
overcome this problem and to minirnize the chance of getting b'stuck" in a local
minimum. In this study, the momentum constant is set to 0.95 after much experimental
training.
Training tolerance represents the allowable variation when the actual output neuron
values are compared to the desired output of the training case. A training tolerance of 0.0
requires an exact match of desired output to actual output that precludes connection
weight updates. A higher training tolerance will allow more variation in the output
values before errors are propagated back through the network. A smaller training
tolerance will provide more accurate results than one training with hi& tolerance, but at
the cost of more training iterations. A training tolerance used in this study is set to 0.001
f i e r much experimental training.
42.32. Training and Testing Examples
The composition of training examples was a consideration in the training process. A
wide range of situations in the training set is thought to enhance the possibility of better-
trained networks.
The size of training set was another implementation issue. The larger the training set, the
richer that set becomes in describing the problem domain and the more likely the network
might be accurate.
A set of testing examples was needed to evaluate the network performance and define the
accuracy of the predictive capacity of the network.
Due to the limited number of completed projects collected, different sizes of training sets
and testing sets were used in the training processes. Based on the lhited samples, the
ANN mode1 was developed through intensive training processes.
Several situations that were considered during the training process included:
Training using different sized training data (randody selected), such as 60%, 75%
and 90% of total samples for training, and three Merent compositions of each sized
training data
i Training using oniy pipeline projects
Training using only refinery projects
4.2.4. Performance Measures of Neural Network Models
The measures of network performance Vary with different applications in the literature.
The ratio of corrective cases in test samples or error ratios are a commody used
measurement used to classify types of neuron network models. Murtaza and Fisher
[ 1994 ] used this measure in a neuron network system for modular construction decision
making. Koster, Sondak and Bourbia [1990 - 199 11 used this measure in a neuron
network to predict banhptcy. Tarn and Kiang [1992] applied misclassification rates as
one of the performance measures for evaluating a network for predicthg bank failure.
The average of operation error, expressed in equation 4.13, is another measurement in
such neuron networks to map the relationships between inputs and outputs. Moselhi,
Hegazy, and Fazio [1991] used this measure in the neuron network for optimum markup
estimation under difFerent bid situations.
(ANN output - Deshed output)
Operation Error (%) = * 100 Desired output
79
A variation of the cross-validation method known as v-fold cross-validation (CVp) is
another approach to rneasure the performance of an AMJ model. Fletcher and Goss
[1993] used this method to estimate prediction risk of their bankruptcy prediction model.
Three mesures we were concemed with in this study and monitored during the training
process were Surn Squared Errors (SSE), Standard Error of Estimate (SEE), and Squared
Coefficient of Determination (R2) of the predicted outputs over testing samples.
SSE is the mathematical criterion (4.12) for optimality over training sets. A lower
SSE does not necessarily irnply good generalization of the network. SSE indicates
how well neural networks fit training samples.
SEE is a rneasure of predictive error variation. SEE is equal to the square root of the
s u m of the squared residual variance (the dflerence between the predicted value and
desired value) and gives an indication of the average absolute error of prediction.
SEE is more directly interpretable than the residual variance.
R2 is a key performance meamernent through the whole training process. The larger
the R2 over the testing sarnples, the better the generalization capability of the network
model.
4.2.5. Generaüzation and Cross-Validation
In back-propagation learning, we start with a training set and use the back-propagation
algorithm to compute the weights of a multilayer network by encodhg as many of the
training examples as possible into the network. The hope is that a network designed in
this manner will be able to generalize. A network is said to generalize well when the
inputsutput relationship computed by the network is correct (or nearly so) for
inputloutput (test data) never used in training the network. It is assurned that the test data
are drawn h m the same population used to generate the training data.
When a neural network Iearns too many specific input-output relations (i.e., it is
overtrained), the network may memorize the training data and therefore be less able to
generalize between sllnilar input-output patterns.
To train networks in the sense that they learn enough about the p s t to generalize to the
fbture, a leaming process arnounts to a choice of network configurations for the data set.
More specifically, the learning process may be viewed as choosing, within a set of
candidate mode1 structures (corQurations), the "best" one accordhg to a certain
criterion.
A standard tool in statistics, known as cross-validation, provides an appealing guiding
principle. As usuai, the available data set is randomly partitioned into a training set and a
81
test set. The training set is used for training a network and the generalization
performance of the network is rneasured on the test set.
4.3. Neural Network Computation SoîYwanr Packages
Since training and tuning an ANN is a computationally intense process, large hi&-speed
cornputer systems are required for the simulations. Mini-workstation cornputer systems
may be suitable for this kind of work. A UNIX system with a speed of about 1 million
cycles per second (36 MHZ) was used to carry out the ANN training job in this study.
There are many commercial software packages available for neurocomputing, ranging in
pnce corn $50 to several thousand dollars and designed for the M X , Macintosh, and
IBM systems. One of those is cailed "Matlab" which is developed by MathWorks Inc.
Matlab typically runs on LTNIX systems and IBM-PCs. The version used in this research
was the 'Matlab' for a U N E system. Braincel is a neurocomputing software package
that works within Microsoft Excel spreadsheets. Other commercial software packages
available for ANN computing include NeuralWorks Professionai II software, developed
by Neural Ware Inc., and NeuroS hel14.1.
Although Matlab performs neuron computing pelfectly for many dserent problems,
customized programming is necessary to configure a neuron network, to initial weights
82
and bias, to input training and testing samples, and to code a batch file for a training
process. A coded batch file (tralliingl .m) is presented as an exarnple in Appendix V.
CHAPTER 5 STUDY RESULTS
5.1. Raw Data Analysis
Data on a total of a hundred and six projects were gathered. The researcher contacted
forty-eight participants to collect these data. Eighteen of the participants failed to return a
completed survey. The responding participants represent about twenty-four
organizations. Fifteen of the participants provided more than one project. The r e m rate
of the survey was 62.5% (30148).
Three projects were excluded fiom the raw data pool because of incomplete answers.
One hundred and three projects were used for data analysis and mode1 development.
Collected projects are categorized into five groups. The distribution of project type in the
oil and gas industry is shown in Table 5.1.
Table 5.1. The Frequency of Project Type
Projects Gathered Pipeline
Refinery
Gas Plant Others* Totals
Frequency 1
36
20 15 32 103
Note: * Others Uiclude cornpressor stations, drillhg wells, etc.
84
A simple descriptive statistic analysis was perfonned. Average score, maximum score,
minimum score, and standard deviation of each of the relative questions were calculated
and the results are listed in Table 5.2.
Table 5.2. Descriptive Analysis Results
---
SURVEY RAW DATA ANALYSIS
1 Outcorne Variances 1 Am.Score Max Min ! S.D. 1
1
-1.1) Rnjecî Estimate Capital Cost (MM) 1 64.95 4600.00 0.53 451.64 lw2) &$kt Acnial Capital Cost 0 1 76.48 5800.00 0.62 569.79
(2) Contingency in capital cost (%) . (3.1) Roject Estimate Duration (Months) (3.21 h iec t Actual Duration CMbnths)
18.2) Numbers of key o ~ o n s involved 1 1 1.71 1'77.00 2.00 24.40
6.34 25.00 0.00 5.33 13.7'7 72.00 200 8.12 -
13.96 84.00 200 8.95
(4) Contingency in scheduie(%)
ûutcome Variance Indicatm (8.1) Complexity
2.75 12.00 0.00 4.22
2.10 3 .00 1 .O0 0.43
ka~ital cost in hn t end as % of estimate caoital cost 1 1.60 2 1.28 0.00 2.95
(9) Piercerrtage of design cwnpleteness ( IO) Capital cost in hnt end (MM)
1 1) Capital cost in Detail k&o 8.68 750.00 0.04 74.51 apital cost in Ml Design as % of estimate capital cost 8.08 32.00 0.55 6.44
84.49 100.00 20.00 22.07 - 0.69 50.00 0.00 5.00
.8) weather &kt (% activity affectedl 1 14.06 - 95.00 , 0.00 17.55
' r
85
As can be seen nom Table 5.2 there was a wide range of cost variances fiom 57% to
180% (percentage of actuai cost to estirnated cost) and a large range of t h e variances
fiom 83% to 173% (percentage of actual duration to estimated duration) in this study.
The project with a 180% cost ovemm was an extreme case and had a very high risk
profile. The project complexity was hi& and new technology was involved. There were
ten key organizations that directly participated in this project and they were
unaccustomed to working together. The percentage of engineering design completeness
at the start of field construction was 60% and the project had poor scope defuiition. Al1
these factors, together with the project changes that occurred, resulted in the significant
cost overrun.
The average cost variance and tirne variance in these studied projects was 102%. On
average, project cost and t h e deviations from the estimates (Approval For Expenditure)
were not significant (+ 2%). The number of projects with cost overruns was forty-six
which present 45% of the total studied projects. The number of projects with tirne
ovemins was seventeen, 17% of the total projects.
5.2. Artificial Neural Nefwork Training and Results
In the developrnent of ANN models, three phases of A N ' training were carried out The
three phases were prelirninary training, phase 1 training and phase II training. Each phase
86
of îmining had its specific objectives. The objectives of preliminary training were to
select a training d e , to detemiine a leaming rate, and a training tolerance, and to screen
transfer functions. The objectives of the phase 1 training were to detemine the transfer
functions which ailow networks to have the best performance in terms of high
generalization capability and to determine what sarnple size allows networks to perform
the best. The purpose of the phase II training was to develop ANN models using critical
input variables and grouped project data. The whole process of the ANN training in this
study is presented in Figure 5.1.
1 Format input data 1
Preliminary Phase Determine training rate, tolerance
I and screen transfer functions t
I -
Phase 1 Vary sarnple size, the number of
hidden nodes and transfer firnctions r Phase II
90% sample Pipeline projects Refinery projects
Figure 5.1. The Process of ANN Training
5.2.1. Fomattîng of Input Data in Training
To make the data comparable, cost variances and t h e variances were normalized. The
percentage of actual cost to estimated cost and the percentage of actual duration to
estirnated duration were used in the model development, instead of using the absolute
actual cost and the actual duration. For exarnple, the estirnated cost of a project was
652MM. The actuai cost of the project was SSOMM. The percentage of actual cost to the
estimated cost (96%) was used in model development.
According to other researchen' experience in ANN training, the magnitudes of each input
data should be as close as possible. In this way, ANN models are easier to train.
Consequently, some input variables, such as the number of key organizations and the
number of permits/approvals, were grouped into five categories. Appendix IV presents
the format of each input variable.
5.2.2. Preliminary Training
The purpose of a prelimlliary experiment was to screen bander fiinctions for fbrther
experimental training. Appropriate trander functions on hidden and output layers should
generate the best performance of the models. As mentioned in Chapter 4, there are three
types of transfer hctions commonly used in back-propagation leaming networks. Any
88
possible combinations of hem were considered. Clearly, there are nine possible
combinations; shown in Figure 5.2. These nine combinations were used in the
prelimuiary training.
Hidden Layer Output Layer
Figure 5.2. Nine of Combinations of Transfer Functions
The resuits fiom the preliminary training concluded that neuron networks that had an
output layer with Logsig and Tansig transfer functions result in no Iearning occurred.
They ody produced outputs of "1". Networks with Pureline output neurons were capable
of leaming in this shidy. Therefore, three cornbinations of transfer functions were used
for m e r training. These are listed below:
Hidden Layer Output Layer Lable
Logsig - Purehe (L ogs ig-Ehre 1 ine)
Tansig - Pureline (Tansig-Pureline)
Pureline - Purehne (Purdine-Pureline)
Figure 5.3. Three Combinations of Transfer Functions
5-23. Phase 1 Training
Further to preliminary training, the first phase of ANN training with varied sample sizes
was carried out in order to select the best combination among the three combinations in
Figure 5.3. The criteria for selection was determined by overall network performance as
determined by the transfer functions of the hidden and output layers. In other words, the
networks generating the largest R* with the testing samples among the three cases were
selected.
Three groups of samples with sizes of 60%, 75%, and 90% of the total usable collected
projects were also considered. These were later used in the second phase to determine
what sample size used in training provides the largest RZ. The number of samples used in
the training were 62, 77 and 93; the number of samples used for testing the performance
of networks were 4 1,26, and 10, respectively.
With each group of training samples, three versions of samples were randody generated.
Therefore, there were nine subsets of samples used in the second round training. The
averages of performance panuneters of three versions of samples were calculated for each
group of sized samples.
533.1. Training and Results with 60% Samples
Three combinations of transfer bct ions were empirically tested through intensive
training. Sixty-two samples were used in training and forty-one for testing the
performance of the model. Various number of hidden nodes (10, 15, ..., 40) were also
used in the training process. Ten thousand was used as the maximum number of
iterations in each training cycle.
Sum Squared Errors (SSE), Standard Deviation (SD), and Coefficient of Determination
(R2) in training and testing were calculated and recorded. The results are shown in Table
5.3, Table 5.4, and Table 5.5.
Table 5.3. SSE (Sum Squared Errors) in Training with 60% Sized Samples
Table 5.3 iilustrates that networks with a Logsig hidden layer and a Pureline output layer
have the srnailest SSEs, regardless of the number of hidden nodes used.
Tansig-Pureline 1 .O4 0.74 0.49
, 0.58 0.6 0.2 1 0.25
Logsig-Pureline 0.52 0.22 0.05 0.13 0.17 0.07 0.03
Hidden Nodes 10 15 20 25 30 35 40
Pureline-Pureline 1.82 1.6 1.91 1.97 2.14 1.53 1.54
The standard deviations of cost variance and t h e variance with the training set and the
testing set were calculated with varying numben of hidden nodes. The averages of SD of
cost variances and time variances were used in the comparison of three combinations of
transfer bctions, presented in Table 5.4.
Table 5.4. SD of Outputs in Training with 60% Sized Samples
Pureline-Pureline Logsig-Ptueline Tansig-Pureline [ Hidden 1 Training ( Testing ( Training 1 Testing 1 Training ( Testing (
Table 5.4 shows that networks with a Logsig hidden layer and Pureline output layer have
the srnaIlest SD over the training samples and have medium values of SD over the testing
samples, regardless of the number of hidden nodes used.
Nodes 10
R2 of COS^ variance and time variance with the training set and the testing set was
calculated with various numbers of hidden nodes. The averages of R2 of COS^ variance
and t h e variance were used in the cornparison of three combinations of transfer
functions, presented in Table 5.5.
Set 0.122
Set 0.163
Set 0.063
Set O. 167
Set 0.090
Set O. 185
Table 5.5 R2 of Outputs in Training with 60% Sized Samples
Pureline-Pureline Logsig-Pureline 1 Hidden 1 Training 1 Testing ) Training 1 Testing
Nodes 1 Set 10 0.49
Table 5.5 illustrates that networks with a Logsig hidden layer and Pureline output layer
Tansig-Purelhe
have the highest R2 over the training samples, regardless of the number of hidden nodes
Training Set 0.72 0.80 0.87 0.86 0.84
Set O. 1 O
used.
Testing Set 0.04 0.08 O. 15 0.09 0.03
5.2.3.2. Training and Results with 75% Samples
Set 0.85
Seventy-seven samples, representing 75% of total sample of projects, were used in
Set 0 .O6
training and twenty-six samples were used for testing the performance of networks.
Various numbers of hidden nodes (10, 15, 20, ..., 40) were also used in the training
process.
Sum Squared Errors (SSE), Standard Deviation (SD), and Coefficient of Determination
(R2) in training and testing were calcuiated and recorded. The r e d t s are shown in Table
5.6, Table 5.7, and Table 5.8.
Table 5.6, Table 5.7 and Table 5.8 show that results in the 75% case are sirnilar to those
in the 60% case. Networks with a Logsig hidden layer and a Pureline output layer have
the smallest SSEs and SD over the training samples and the highest R2 over the training
samples.
Table 5.6. SSE ( S m Squared Erroa) in Training with 75% Sized Samples
Table 5.7. SD of Outputs in Training with 75% Sized Samples
Tansig-Purelime 1.908 0.91 1 1.678 0.839 0.554 0.537
0.250
Logsig-Pureline 0.678 0.328 O. 123 0.399 0.084 0.036
0.183
Hidden 10 15 20 25 30
35 40
Pureiine-Pureline Log sig-hireline Tansig-Pureline
Pureline-Pureline 2.335 2.6 19 2,284 2.582 2.304 2.389
, 2.433
Testing Set
0.276
Training Set
0-1 10
Testing Set
0.232
Training Set
0.060
Testing Set
0.151
Hidden Nodes
10
Training Set
0.124
Table 5.8. R2 of Outputs Ui Training with 75% Sized Samples
5.2.3.3. Training and Results with 90% Samples
Pureline-Fureiine Logsig-Pureline Tansig-Purelhe
Ninety-three samples were used in training and ten samples were used for testing the
performance of networks. Various numbers of hidden nodes were also used in the
training process.
Sum Squared Errors (SSE), Standard Deviation (SD), and Coefficient of Determination
(R2) in training and testing were caiculated and recorded. The results are illustrated in
Table 5.9, Table 5.10, and Table 5.1 1.
Table 5.9, Table 5.10, and 5.1 1 illustrate that networks with a Logsig hidden layer and a
Pureline output layer have the srnaIlest SSE and SD over the training set and have the
highest R2 over both the trainhg set and the testing samples. We also can see that in
average of seven cases of hidden nodes, the R2 over the testing set increase and the R2
Testing Set 0.0 1 0.05 0.08 0.17 0.03 0.15 O. 16
Training Set 0.52 0.79 0.60 0.8 1 0.84 0.88 0.95
Testing Set 0.13 0.04 0.07 0.03 0.02 0.12 0.06
Training Set 0.86 0.9 1 0.97 0.9 1 O .99 O .99
Testing Set 0.05 0.07 0.06 0.09 0.05 O .O4
Hidden . Nodes
10 I S 20 25 30 35
Training Set 0.39 0.37 0.42 0.37 0.4 1 0.41
0.07 1 0.96 40 0.39
over the training set decrease as the size of training samples increases (see Table 5.5,
Table 5.8, and Table 5.1 1).
Table 5.9. SSE (Surn Squared Errors) in Training with 90% Sized Samples
Table 5.10. SD of Outputs in Training with 90% Sized Samples
Tansig-Pureline 2.545
Logsig-Pureline 1.105
Hidden Nodes 10
Pureline-Pureline Logsig-Pureline Tansig-Pureline
Purelhe-Pureline 2.706
Testing Set
0.16 1 0.160
Testing Set
O. 174
0.167
Hidden Nodes
10
IS
Training Set
0.1 16 0.093
Testing Set
0.095 0.099
Training Set
O. 120
0.133
Training Set
0.074 0.050
Table 5.1 1. R2 of Outputs in Training with 90% Sized Samples
5.2.4. Cornparison among Three Group Samples
Pureline-Pürebe Logsig-Pureline Tansig-Pureline
In section 5.2.3, the results fiom ANN training with three group samples and three
combinations of ûansfer functions are demonstrated. It is concluded From this
information that networks with a Logsig hidden layer and Pureline output layer
performed the best among the three combinations of transfer function. Networks with
twenty-five hidden nodes had the largest R2 over the testing samples. So, the
combination of Logsig-Pureline transfer function with twenty-five hidden nodes was
selected for fûrther comparison among the three shed samples.
R2 of cost and time variances over testing samples presents the generalization capacity
(cross-validation) of the predictive models. The larger the R2 over testing samples, the
better the network model. In the context of the rest of this chapter, SD and R2 were iisted
Training Set 0.45 0.64
0.69
0.82
0.73 0.84
0.9 1
Testing Set 0.2 1 0.14
I
O .O9 O. 14 O. 10
0.1 1
0.18
Training Set 0.73
0.88 0.86
0.92
0.97 0.94
0.99
Testing Set 0.25
0.20
0.3 5
0.27 0.24
0.22
0.30
Hidden Nodes
t O
15
20
25
30
35
40
Testing Set 0.14
0.17
0.25 0.38 0.25
0.32
0.30
Training Set 0.40
0.36 0.38
0.33
0.36 0.35
0.32
97
only with testing samples. Table 5.12 presents the sized samples verses SD and R2 of the
testing samples.
Table 5.12. SD and R2 of Outputs VS. Sized Samples
1 Sued Sam~les 1 Standard Devirtion 1 R* 1
It was obvious that 90% sized sarnple had the smallest SD and the largest R2. A
conclusion can be made that the more samples used in training, the better the
performance of neural networks. There is no maximum limit of training set size. It is
important to collect a minimum amount of samples for training in order to reach an
acceptable threshold of prediction accuracy. Recent research [Bode, 19981 showed that
the marginal contribution of additional training data decreases with growing ûaining set
sizes.
5.2.5. Defining Critical Input Variables Using Multiple Linear Regression
Eighteen predictors treated as independent input variables were used in the fist two
phases of ANN training. It was established that these predictors were highly correlated
rather than independent. For example, project type will influence the location of the
project. Pipeline projects are mostly located in remote/suburban areas. Observations
98
from personal interviews with practitioners in industry indicated that fast-tracking
approaches are rarely applied on projects with brand new technologies. It would be too
risky to do so. There was another learning from the survey that to some degree, design
completeness of a project is related to how familiar the project team is with the
technologies used and their experience with similar projects.
From an practical point of view, the fewer the number of subsets of available good
predictors, the less time needs to be spent gathering information, and hence the more
economical modeling. This holds tme as long as the subset of predictors can explain
nearly as much of the cost and tirne variances as is explained by the entire set of
predictors.
It is necessary to dehe the most critical indicators to project cost and time variances.
This also reduces rnuch of the ANN training time. As known in statistics, a stepwise
solution enables us to include only those predictors that add significantly to predictive
power. A forward regression approach was considered and used in defining the
significaat predictors.
5.2.5.1. Foiward Regression
In a forward regression, the multiple regression equation is built one step at a time by
sequentially addinp predictors to the equation. The first predictor for entry into the
equation is the one with the largest correlation with the dependent variable (the highest
R2).
To determine whether a variable is entered into the equation, the F value was calculated
and compared to an established critenon. For instance, in this study, PIN (probability of
F-to-entry) with a value of 0.05 was used. In this case, a variable enters the equation only
if the probability associated with the F test (PM) is less than or equal to the default 0.05.
If the criterion is met, the variable is entered into the equation and the procedure is
repeated. The variable with the largea partial correlation is the next candidate. Choosing
the variable with the largest partial correlation in terms of absolute value is equivalent to
selecting the variable with the largest F value. The procedure stops when there are no
variables that meet the entry criterion.
The SPSS software package was used and pefiormed the forward procedures. There were
two dependent variables, cost and time variance. For each dependent variable, a forward
regression was perfomed.
5.2.52. Forward Regression with Dummy Variables
5.2.5.2.1. Dummy Variable
Because there were categorical variables (discretion variables) involved which had no
quantitative meaning, these variables had to be converted into a series of dichotomously
scored "dummy" variables. The number of dummy variables created was equal to the
number of categones of the original variables and each case was scored O or 1 on each
durnmy variable. Scores of O were used to indicate lack of membership in the category
represented by a dummy variable; Scores of 1 showed that a case was a member of that
category. For example, the original variable 'complexity' was converted to three dummy
variables representing three categories of complexity with 'high', 'medium', and 'low'
respectively. When a case of 'high' occurred, the score of the first dummy variable was
1.
Once independent variables (contûiuous and dummy) were refomed (see Appendix IV
for details), two separate forward regression analyses were carried out using the entire set
of one hundred and three samples. One forward regression analysis was for cost-
variance; the other was for the-variance. Two lists of significant predictors were gained,
shown in Table 5.13 and Table 5.14.
Table 5.13. "Dummy", Discretion Variables, R2 and SD of Cost Variance
Note: See Appendix IV for the details of the description of durnmy variables.
Dum. V.
' V 3 3
V 16-3
V 1 7-2
v2-1
Figure 5.4. DEerence of R2 VS. SequentiaUy Added Predictors to Cost Variance
Dis. V. V7 V3 V6
V16 V 1 5
7 I
Description
‘‘hi&"
"Unit Price"
"Established in the industry but new to your Org." ,
As can be seen in Table 5.13, nsk factors, such as project cost spent in fiont-end and
detail engineering, hi& complexity, 'Unit prke' contract, years of experience, new
technology and project location with 'Urban', add signincantly predîctive power for cost
variances prediction. Figure 5.4 presents the relative significance of the predictors to the
"Urban"
Description Cost in Detail Design
Compiexity
Cost in Front End
Contract type
Years of Expenence
Level of Technology
v2
R2
0.24
0.3 1 0.40
0.44
0.49
0.5 1 I
SD 0.16 1
0.153 0.143
0.139 0.1 34 0,132
1 Project Location 0.53 0.129
102
cost-variance. 'Cost spent in the fiont end stage' is the most important variable to cost
variances. 'High Complexity' is the second most important variable to cost variance.
'Unit-price Contract' and 'Years of experience' are right in the middle. 'Level of
technology' and 'Project location ' are less important than othen. Figure 5.4 also
indicates that 'Cost spent in detail design', 'Complexity' and 'Cost spent in fiont-end' are
the top three predicton which rapidly increased the R* of cost-variance.
Table 5.14 "Dummy", Discretion Variables, R2 and SD of Time Variance
Dum. V.
v4-3 v i s
v 16-3 VI-2
V18-3 v4-5
V2-2 v3-3
VI-1
v 12-4
vi2-1 2 - 1
V 1 7-3
Note: See
Dis. V.
v4 VI V16 v1
VI8 v4 V2 v3 VI V5
VI2 v12
v 2 V13 V 1 7
of the
Description "From 15 to 30"
"Othen"
"Unit Price"
"Pipeline"
'5 25% activity affected"
"No's of Key Org. >50"
"Suburban"
"High"
"Gas Plant"
"From 15 to 20"
"From O to 5"
"Urban"
"New to the indutry but used in other industry"
Appendk IV for the details
SD 0.115 0.111 0.108
0.104 0.098 0.095
0.093 0.091
0.090 0.087
0.085 0.083 0.080 0.077 0.075
Description No's of key Orgs.
Project Type
Contract Type
Project Type
Weather
No's of key Orgs.
Project Location
CompIexity
Project Type
Design Completeness
No's of Permits
No's of Permits
Project Location
Scope Defined
Level of Technology
description of dummy
1 R2 0.12 0.19
0.24 0.30 0.39 0.43 0.46 0.49 0.51 0.54
0.57 0.60
0.63 0.65 0.68
variables.
1 O3
We c m tell from Figure 5.5 that 'Weather' with 'over 25% activity affected' adds the
most to the R-square of time variance. The second most significant variable is the
'Project typeT with 'others'. 'Complexity' ranked lower in tirne variance compared to
cost variance.
.-
Figure 5.5. Difference of R2 vs. Sequentiaily Added Predictors to Time Variance
5.2.5.2.2. Definhg Input Variables
After compiling the two lists of predictors for cost-variance and the-variance
respectively, a List of thirteen variables having sigoificant importance to cost and time
variances was obtained (see Table 5.15).
Table 5.15 illustrates that 'Project Location', 'Complexity', 'Contract Type', and 'Level
of Technology' are signincantly important to both cost variances and time variances.
1 O4
Table 5.1 5 clearly shows that there are more factors influencing time variances than cost
variances. In other words, time variances rnay result from more complex reasons than
cost variances.
Table 5.1 5. Input Variables in the development of ANN Models
Note: * donates that the variables marked were significant to cost variances or to time variances.
As known, the fast-tracking approach c m speed up project construction but usually
causes increasing cost to get shorter duration. Therefore, it may be argued that 'Design
Completeness' should have significant importance to cost variations. It has to be
understood that each of the factors has a certain level of importance to cost-variance. The
results fiom the regression d y s i s showed 'Design Completeness' did not add
significantiy to the predictive power of cost-variance. Finally, Thirteen varîables listed in
Cost Variance
*
n
*
*
Description
Project Type Project Location
Complexity No'sofkeyorganizations
Design Completeness Cost in Front End
Cost in Detail Design No's of Permits Scope Defined
Years of Experience Contract Type
Level of Technology Weather
ID
1
Time Variance
*
* * n
Variables
v l 2 3 4 5 6 7 8 9
I O Il 12 13
v2 v3 v4 v5 v6 v7 VI 2 VI 3 VI 5 VI 6 VI 7 VI 8
105
Table 5.15 were selected to be input variables in the second phase of the development of
ANN model.
5.2.5.3. Forward Regression without Dummy Variables
The second way to define the cntical attributes to cost and time variances was the use of
the discretion variables. Artificial neural network approaches can process with both
binary-value and continuous value. it is not necessary to refom discrete variables
(category variables) to dumrny variables (binary-value). The real value of each variable
could be simply used.
Forward regression analysis was performed using SPSS. Two sets of critical indicators to
cost and tirne variances were obtained, shown in Table 5.16 and Table 5.17. The
remaining indicators were not listed.
Table 5.16. Original Variables, R2 and SD of Cost Variance
SD 1
0.161
0.153
V6
R* 0.24
0.3 1
Original Variable
v 7
V3
Description Cost in Detail Design
Complexity I
Cost in Front End 1 0.3 7 0.147
1 O6
From Table 5.16 and Table 5.17, we can derive the following table (Table 5.18) which
illustrates the significant predictors to codtirne variances or both. These ten predictoa
were selected to be used in th5 second phase of the development of the artificial neural
network models. Table 5.1 8 clearly shows that project complexity is the only variable
that has significant impact on both cost and time variances.
Table 5.17. Original Variables, Et2 and SD of Tirne Variance
Table 5.18. input Variables in The Development of ANN Models
Original Variable
v 4
V18 v1 VI1 V3 v 5
V2 V15
R2 0.12
0.17
0.2 1
0.23 0.25 0.27
0.29 0.30
Description No's of Key Orgs.
Weather
Project Type
Priority: Cost
Comp Iexiq
Design Completeness
Project Location
Years of Experience
SD 0.1 1s
0.1 13
0.1 11
0.1 10
O. 1 09
0.108
0.107
O. 1 06
Time Variance *
t
*
*
Cost Variance ID
1 2 3 4 5 6 7 8 9 10
Variables
v 1 v2 v3 v4 v5 v6 v7 VI 1 v15 v18
Description
Project Type Project Location
Corn plexity No's of key organizations
Design Completeness Cost in Front End
Cost in Detail Design Prïority: Cost
Yearç of Experience Weather
1 O7
As we see in Table 5.18, only one variable 'Priority: C o d was not in Table 5.15. The
test were in Table 5.15. It indicated that there was a consistency between the two
methods. More interesthg was to see the results fiom ANN training using these two sets
of input variables.
5.2.6. Phase II Training - Using Critical Input Variables
After we determined two sets of input variables, ANN training processes were then
undertaken using grouped project data. The total project data were grouped into three
sets listed as the follows:
W Total projects (Case 1): one hundred and thee projects (ninety-three for training and
ten for testing)
Pipeline projects (Case II): thirty-six pipeline projects (thirty-two for training and four
for testing)
Refinery projects (Case III): twenty refinery projects (sixteen for training and four for
testing)
The configuration of the ANN models is kept the same as in the ANN training ushg
eighteen variables except that the number of nodes on the input layer was changed
because the number of input variables changed. The number of hidden nodes needed to
108
be determined through a training process. The standard deviations and R2 of cost and
time variance fkom predicting testing data are presented in Table 5.19.
Table 5.19 shows that the performance of ANN models trained using project data fiom
one type of project are better than those using al1 available project data which has a broad
range of project types. The reason is that, in the pool of studied projects, there are several
different types of projects, such as pipeline, gas plant, refinery, etc., each of which has
different risk patterns. Decisions, risk impacts and project outcomes differ significantly
from one project type to another. These differences cause difficulties for the leaniing
processes of neural networks and result in lower performance of ANN models.
The standard deviations of error of predicted cost variance are larger than for time
variance. In the separate data cases for pipelines and refinenes, the R2 of t h e variances
are larger than for cost variances in the three cases of input variables. This means that the
capability of ANN models to predict duration variance is better than for cost variance.
But in case 1, the R2 of COS^ variances are larger than those of t h e variance. The standard
deviations of cost variance of refhery projects are larger than for pipeline projects. This
may be due to the greater complexity of refmery projects than pipeline projects.
1 O9
We cm see that the pefiomance of ANN models with ten variables is better than those of
other models throughout case 1, case II, and case III. It seems that it is not necessary for
ANN rnodels to use 'dummy' variables in the determination of critical indicaton.
5.3. Penonnance Comparison between N e u d Network and Multiple Linear
R e g m i o n Analysis
In this section, performance comparison between ANN rnodels and multiple linear
regression models is presented. The comparison use the Case I sarnple. The same group
data and independent variables were used to build up regression models. Standard
deviation of error and R2 with testing data were calculated and the results are illustrated
in Table 5.20.
Table 5.20 shows that the A m ' s performance measure, R' is much larger than that of
regression models. Predicted values fiom ANN models highly correlate the desired
values. The generdization capability of ANN models is superior to multiple linear
regression models. It means that the predictive accuracy of ANN models is higher than
multiple linear regression models. ANN models are supenor to the multiple linear
regression models in predicting project cost and îirne variances.
Table 5.20. Comparison between ANN Mode1 and Multiple Linear Regression
CHAPTER 6 CONCLUSIONS & RECOMMENDATIONS
6. f . Conclusions
The effectiveness of the current nsk analysis techniques is heavily dependent upon
experts' personal experience and judgment. The weaknesses of these techniques have
limited their applications in complex situations in which no mathematical models can be
applied and interactions between nsk factors cannot be quantified. This weakness
necessitated the adoption of a new approach to assess project cost and time risks.
This research studied the potential applications of artificial neural networks in the
assessrnent of project risks in the early stages of a project. Neural networks were used to
capture the relationships among risks, project characteristics, decisions, and outcomes.
Intelligent models were then developed and tested. They can be used to predict project
cost and t h e variations.
The research remlts show that artificial neural nehntork technology surpasses
conventional models such as multiple linear regression andysis, which is a cornmonly
used method to build predictive models. The practical application of ANN technology in
project risk analysis is promising, especiaily in the fiont-end stage.
113
This research made significant progress in the quantitative assessment of project cost and
time risks using artincial neural network technology. More complicated ANN models
with a quality development process and high-level performance measures are applied in
this research.
McKim [1993] studied quantitative assessment of project cost overruns using artificial
neural network technology. Four risk factors (contractor, architect, location and size)
were considered. Twenty projects were used to develop an ANN model. Standard
deviation of error was used to measure the performance of the ANN model. A standard
deviation of error of cost overruns fiom his ANN model was 3.6.
In this research, the largest standard deviation of error of cost ovecruns fiom ANN models
was 0.396. This is about a 90% improvement in standard deviation of error compared to
the results fkom McKim's model. It can be explained that McKim's model was too
simple to explain the variations of cost overruns. More important risk factors should be
considered in the ANN models.
Signifïcant risk factors to cost variances and time variances were identified in this study.
Project type, project location, cornplexity, number of key organizations involved in a
project, design completeness, cost as the number one project priority, years of expenence
in similar projects that a project leader(s) has and weather were identifcd having high
114
correlations with t h e variation. Cost spent in detail design, complexity and cost spent in
the front end engineering phase were identified to have high correlations with cost
variation. Intuitively, cost as the number one project prionty has important impacts on
cost variance rather than on time variance. But if you have a limited budget you might
spend more time in the engineering phase to fhd a cost-effective approach in your
project. There is a tradesff between money and time.
Recent research [Pedwell, Hartman and Jergeas, 19981 concluded that the fast-tracking
approach produces less project definition at the start of construction and reduces the total
project duration but usually causes cos? increases to achieve this shorter duration.
Therefore, design completeness should have significant impacts on cost variances.
However, in this study, design completeness was not identified as having high correlation
with cost variances.
This study's results show that project complexity is the most significant attribute to cost
variation and tirne variation. In this study, project complexity refers to technical
complexity of a project. This is a very subjective term. Here, project technical
complexity hcludes elements such as the accessibility, technical requirements and
Limitations, numbers of major rotating equipment, and specialty of materials. It can be
concluded that reducing or managing these complexity factors in a project at the fiont end
115
planning stage may resdt in a greater probability of successful implementation of the
project.
Overall, projects will have a greater chance of success, in tems of 'within budget' and
'on time', when project managers direct more effort into managing the identified
important factors during project planning in the Front end phases.
Other conclusions fiom this research include the following:
An artificial neural network is able to caphue the rkk patterns of projects by learning
fiom histoncal project samples and to generate a reasonable prediction of project cost
and tirne variances.
Artificial neural network technology is superior to conventional multiple linear
regression analysis in building predictive models of project cost and time variance
behavior.
ANN with stepwise regression andysis provides more accurate estimates of project
cost and time variations than ANN on its own. It aiso significantiy reduces the
training t h e and increases the training efficiency of the networks.
116
0 The larger the sample size used in the development of ANN models, the more
accurate the ANN model is likely to be.
0 ANN model generalization would be improved if similar sarnples were used, for
instance, similar projects and projects collected fiom one organization.
This research built a rational base for developing a decision support system to assist
project managers (decision makers) in better decision making.
The scope of this research is delimited to the project implementation phase and only
project intemal risk factors were modeled. Therefore, the use of the ANN models is
delimited.
The project operation phase is an important phase to a project. The ANN models will be
improved if project operational risk factors are included. Project extemal risks were
ignored in the development of the ANN models. The external factors could be related to
elements such as oil price, labor market, and project revenue cashflow. These factors
have iduences on projects in terms of cost and t h e . Future research in project external
risks in the development of ANN models is recommended.
117
The resuits fkom this study were positive, but there are some limitations. The major one
is the existence of bias in the data. Since the data was collected from different
organizations and different individuals, the data would be not consistent in subjective
issues. Different individuals would have different expenences and therefore different
opinions about similar occurrences or situations. Inconsistent data could introduce a
certain amount of noise into the ANN models. Solutions to improve data consistency
include the following :
Elirninate subjective issues.
Provide clear explanations on what information is being collected.
Focus on one organization and one group of project personnel if possible.
6.2. Recommendations
ANN technology shows promise for application in the field of project nsk analysis. The
issues of developing of ANN models have to be carefully considered. ANN training is a
time consuming process. The determination of ANN configuration is the most diacult
portion of the work. Recommendations firom this research for the development of ANN
models include the foilowing:
118
The paradigm and configuration of neural networks have to be congruent to the nature
of the problem. The panuneters of the networks, such as leaming rate, learning
tolerance, and rnomentum constant, have to be defined through an intensive training
process.
ANN training time can be saved if critical input variables are used. Stepwise
regression analysis is an approach to define critical input variables.
During the training process, both over-fitted and under-trained phenornena were
observed. Over-fitting the training samples happens when ANN models are over-
trained (too many iterations allowed or too many hidden nodes). This results in ANN
models losing their generalization capabilities. Not enough iterations of training
could remlt in inadequate Iearning of ANN models. No f o d a exists to calculate
the exact number of iterations and the number of hidden nodes that should be used in
ANN training. Therefore, the number of iterations of ANN training and the number
of hidden nodes have to be adjusted through the extensive training processes.
The performance measures of ANN models have to be set up in advance. The
m e m e s c m be ditferent h m one problem to another. The author strongly
recommends that cross-validation (R2 in the testing samples) be used as a
performance measure of predictive models when developing them.
119
Sarnple sizes used in training and testing are also important. Data sample size will
significantly affect the performance of ANN models. The Iarger the sample, the
better the performance is likely to be. A minimum size of sample for training is
necessary to reach an acceptable threshold of prediction accuracy.
It is recommended that an organization-wide database, continuously updated be built to
capture al1 records of completed projects in the organization. The database will provide
solid and rich data for performing project risk analysis and predicting the outcornes for
proposed projects. This will definitely benefit the project management tearn.
The recommendations for the practical Mplernentation of the ANN models in industry
would include the following:
r Transfer the ANN models developed (Le., ANN configurations) from the CTMX
system to PCs using IBM-PC version of Matlab software package if necessary.
Design and program the interfaces between users and the ANN models including data
entry, presentation of resuits, and analysis of results.
6.3. Further Research
This work has opened a wide research potential and made significant progress in the
project risk analysis domain using a novel approach - artificial neural network
technology. More work will need to be done to improve and implement the developed
ANN models.
Further research work should include the:
Use of the backward approach to fmd out the relative importance of the independent
variables to the cost and time variances ushg the developed ANN model.
Consideration of extemal risk factors.
Use of cluster analysis to group projects based on a similarity measure called
"resembtance coefficient". The smailer the value of the coefficient, the more similar
the projects are. Using more similar projects to train neural networks could results in
better performance of the models and improve the accuracy of prediction.
a Performance of the ANN models wodd be much better if the samples used in the
training process are fiom a single organhtion. This is probably because project
121
performance maintains some similarity within the same organization. Also, risk
factors can be identified more specifically to the organization. This would increase
the predictive accuracy of the models for project cost and time variations.
Use of sensitivity analysis on ANN models to establish how the performance of the
ANN model changes as one of the inputs changes. This will assist project managers
to screen decision alternatives and choose the best one.
Establishment of an industry expert panel compnsing expenenced project mmagers
to test the predictive capability of ANN models against the experts. Experts will be
asked to provide predictions of project outcornes in ternis of cost and time variances.
The projects used in the test are a small sample of the data pool.
6.4. Contribution of the Thesis to the Body of Knowledge
This research has demonstrated the feasibility of applying artificial neural network
technology to the development of predictive models of project cost and tune variances in
the early stages of projects. The artificially intelligent predictive models surpass multiple
linear regression models in the study area This research suggested the development
procedures of the network models and addresses the most sigoificant issues surrounding
neural network model development.
122
For capital project planning and control, an accurate estimate of capital project costs is
most important, especidly in the front-end stages. This research provides a base to
predict project cost and time variances. This research explores the most significant
factor-project complexity- that correlates to project cost and tirne variances. Other factors
af5ecting cost and tirne variances are also identified. The relationship between factors
and project cost and time deviations are established within the neural networks. More
focus on these factors in the decision making process in the front end stages will increase
the possibility of the success of capital projects in tems of 'within budget' and 'on time'.
This research provides the basis of an ANN model to assess the project cost and time
risks. Based on such an ANN model, a decision making support tool can be developed to
help project managers screen the potentiai options and make better decisions.
The research proposed a different approach to evaluate project risks and to capture project
risk patterns. ANN7s ability to leam would help practitionea capture an organization's
knowledge on projects and to create and irnprove an organization's learning.
There are two practical uses of the ANN models in risk analysis and management. They
are the foiiowing:
Provide a rational base for a contingency plan when a
for Expenditure (AFE). The ANN models typically
123
project is going for Approval
help owner organizations to
assess not only an individual risk impact but also the impact of a set of risks on
project cost and time at the early stage of a project. The models are used when initial
estimates of cost and duration of the project are established. Project nsk impacts on
project cost and tirne are then evaluated to determine a contingency. By using the
ANN models, project cost and time variances will be quickly determined by assessing
the individual nsk impacts on the project cost and time. The total variances could
form the basis of a contingency plan for the project.
Define cost and time variances resulting from individual risk using the ANN models
and help practitioners to establish inputs for perfonning conventional risk analysis,
such Monte Carlo simulation. It will reduce expert's personal opinion and judgment
on the risk impacts as inputs to Monte Carlo simulation.
Developing a neural network mode1 that simuitaneously considers project cost and time
variances is a previously unexplored area of study. The methodology of, and the results
from, this research will provide valuable references for sirnilar friture research work.
REFERENCES
Bansal, A., Ka&an, R. J. and Wei@ R. R., Comparing the modeling performance of
regression and neural networks as data quulity varies: a business value approach,
Journal of Management Information Systems, Summer 1993, Vol. 1 O, No. 1, pp. 1 1-32.
Basheer, 1. A. and Nadar, Y. M., Predicting Dynamic Response of Adsorption Columns
with Neural Networks, Jouml of Computing in Civil Engineering, Vol. 10, No. 1,
January, 1996.
Bataineh, S ., An expert cornputer- based system for munuging complex decision-making
systerns, Journal of Cornputer Information Systems, Winter 1995- 1996, pp. 10 1 - 105.
Bode, Jhgen, Neural Networkr for Cost Estimation, Cost Engineering, January 1998.
Bowers, J. A., Data for project risk analyses, International Journal of Project
Management, 1994 12(1), pp.9-16.
Caudill, M., Neural Network Training Tips and Techniques, AI Expert, January 199 1.
Cooper, Dale and Chaprnan, Chns, Risk Analysis for Large Projects, John Wiley & Sons
Ltd., 1987.
Cooper, Donald R. and Emory, C. William, Business Reseurch Methoh, Richard D.
Irwin Inc., 1995.
De Grooî, C. And Wurîz, D., Anaiysis of univariate time series with connectionist nets: a
case study of two clarsical examples, Neuro Computing, 3, pp 1 77- 192, 199 1 .
125
Demuth, Howard and Beale, Mark, Neural Nemork Toolbox, The MATHWORKS Inc.,
1995.
Denton, J. W., Sayeed, L. and Perkins, N. D., Neural Networks to CZussz~ Employeesfor
T a Purposes, Accting., Mgmt. & Mo. Tech., Vo1.5, No.2, pp. 123- 138, 1995.
Dey, Prasanta, Tabucanon, Mario T. and Ogunlana, Stephen O., Planning for Project
C o n ~ o l Through Risk Analysis: A Peîroleum Pipeline-laying Project, International
Journal of Project Management, 12 (l), pp. 23-33, 1994.
Diekmann, J.E., Risk Analyss: Lessonsfiom Artifcial Intelligence, International Journal
of Project Management, Vol. 1 O, No.2, May 1992.
Doherty, N.A., Corporate Risk Management, McGraw-Hill Inc., New York, NY, 1985.
Duggal, S. M. and Popovich, P. R., Practical applications of neurai network in business.
Journal of Cornputer Information S ystems, winter 1992- 1993.
Dutta, S. and Hekhar, S ., Bond-rating: a non-conservative application of neural
networks, Proceedhgs of the IEEE International Conference on Neural Networks,
Vol.I1(1988), San Diego, CA, pp.443-450.
Eberhart, R. C. and Dobbins, R. W., Neural Network PC Tools, Academic Press, 1990.
Federle, M. 0. and Pigneri, S. C., Predicrnte Mode1 of Cost Chemins, AACE
Transaction, 1993.
126
Fletcher, D. and Goss, E., Forecasting with neural networkr: an application using
bankruptcy data, Information & Management, 24 (1 993) pp. 159-167.
Hecht-Nielsen, R., Neurocompziting, Addison-Wesley Co., New York, 1989.
Koster, A., Sondak, N. E. and Bourbia, W., A business application of artifcial neural
network qstem, The Journal of Computer Information Systems, winter 1990- 199 1, pp.3-
9.
Johnston, S.J., Neural network are muking inruad: technology has found many
practical applications, Inforworld, July 8, 199 1, pp. 13- 16.
McKim, R. A., Neural networks and identifcation and estimation of risk, AACE
Transactions, 1 993, pp.5.1-5.1 O.
Merrow, W. Edward and Yarossi, Mary Ellen Assessing Project Cost and Schedule Risk,
1990, AACE Transactions.
Morris, P.W.G., and Hough, G.H., The Anatomy of Mq-or Projects, John Wiley and Sons,
New York, 1995.
Moselhi, O., Hegaq, T. and Fazio, P., Neural networks as tools in consîruction, Journal
of Construction Engineering and Management, Vol. 1 17, No.4, December 199 1, pp.606-
625.
Murtaza, Mirza B. and Fisher, Deborah I., Neuromodex - Neural Network System for
Modula7 Construction Decision Making, Journal of Computing in Civil Engineering,
Vol. 8, No. 2, Apnl, 1994.
127
Mustafa, M. A. and Al-Bahar, J. F., Project Risk Assessrnent Using the Analytic
Hierwchy Process, IEEE Transaction on Engineering Management, Vo 1.3 8, No. 1, Feb.
1991.
Osyk, B. A. and Vijayaraman, B. S., Integrating expert systems and neural networks,
Information System Management, Spring 1995.
Papageorge, Thomas, Risk Management for Building Profesionais, 1 98 8, R. S. Means
Company Inc.
Pedwell, K., Hartman, F. and Jergeas, G., Project Capital Cost Risks and Contracting
Strategies, Cost Engineering, January 1 998.
Pedwell, K., Liu, X. and Hartman, F., Computer Models for Assessing the Probability of
Achieving Time and Cost Targets, Proceedings of The Third Canadian Conference on
Computing in Civil and Building Engineering, Montreal, August 1996.
Pike, R.H. and Ho, S.S.M., Risk Analysis in Capital Budgeting: Barriers and Benefis,
Omega, Vol.19, N0.4, 1991, pp.235-245.
Simister, Steve, Usage and Benefis of Project Risk Analysis and Management,
International Journal of Project Management, 1994 12(1), pp.5-8.
SLicher, A. W. R., Vakalis, P. and Singh, G., An innovative approach to training neural
networks for strategic management of comtruction fim, CML-COMP PRESS,
Edinburgh, UK, 1995.
128
Sohl, J. E. And Venkatachalam, A. R., A Neural Neîwork Approach to Forecasiing
Mode1 Selection, Information & Management 29, 1 995, pp.297-303.
Suh, Y. H. and LaBane, J. E., An applicaiion of artifcial neural network models to
porfolio selection, Journal of Cornputer Information Systems, Fdl 1995, pp.65-73.
Tal, Benny and Nazareth, L inda, Artifcial Intelligence and Economic Forecast ing
Canadian Business Economics, Spring 1995, pp.69-74.
Tarn, K. Y. and Kiang, M. Y., Managerial applications of neural neiworks: the case of
bank failure predictions, Management Science, Vo1.38, No.7, July 1992, pp.926-947.
Touran, Ali and Wiser, Edward P., Monte Carlo Technique with Correluted Random
Variables, Journal of Construction Engineering and Management, Vol. 1 1 8, No.2, June
1992.
Wasserman, P. D. and Schwartz, T., Neural Nehvorks, Part I. IEEE Expert, 2:4, 1987, pp.
10-13,
Wilson, R.L., The strategic organizational use of neural networks: an exploratory study,
unpublished dissertation, University of Nebraska-Lincoln, 1990.
Wirba, E.N., Tah, J. H. M. and Howes, R., Risk Inierdependencies and Natural Language
Compututions, Engineering, Construction and Architectural Management, Vo1.3, No.4,
Dec. 1996.
129
Wong, B. K. and Monaco, J. A., &ert qstem applications in business: a review and
analyss of the Ziterature (1 9 77- 1993), Infornation & Management 29( 1 999 , pp. 1 4 1 - 152.
APPENDIX 1. SURVEY RESULTS ON CURRENT USAGE AND BENEFIT OF RlSK ANALYSIS TECHNIQUES
1. Suwey Questionnaire on Current Usage and Benefits of Risk
Analysis Techniques
Introduction
Risk analysis and management has received increasing attention fiom project
management practitioners. The purpose of this study is to investigate the:
current use of risk analysis techniques by industry in the Calgary area,
what risk analyses are being perfomed,
what cornputer software packages if any are being used to carry out the
anaiysis, and the reasons for and benefits arising nom its use.
The research results will be released to each participant. The results will provide project
management practitioners with state-of-art information on rîsk analysis and management
in indusîry.
0 Questions
1. Yom Name: Te1 #: Mailing Address:
2. Willing to participate another main survey: Yes No
1 . Use "4s" to identiQ the size of projects that risk analysis techniques should be used
by project management:
1 Most use 1 Better to use 1 No need to use 1
1 I 1
I Over 20,000,000 I I I
4. If your Company uses risk analysis techniques, what were the reasons:
(Circle one or more items)
(1). Client request
(2). Internai Policy
(3). Persona1 use
(4). Required by other people in own organization
(5). ûther (speciQ):
Note: I = Less usefull: 2=Neutral; 3 = Usefil; J= Very Useful.
133
6. Check out one or more cornputer software package(s) in risk analysis used in your
organization:
7. Select one or more of following benefits of using risk anaiysis techniques:
Cornputer Software 1. Ccystal Bal1 (works with Excel, Lotus)
3. Risk + (Works with MS-Project V.4.0)
5. Monte Carlo (Works with Primavera Proiect Planner)
7.Other(speciQ):
(1). Allows the formulation of more realistic plans, in terms of cost estimate and timescale;
(2). Gives an incnased understanding of the risks in a project;
(3). Allows the assessrnent of contingencies that actually reflect the risks;
Used
(4). Facilitates greater, but more rational, risk taking, thus increasing the benefits that can be
gained from risk taking;
(5). Identifies the party best able to handle a nsk;
Computer Software
2. @Risk (for Excel)
4. @ Risk for Project
6. Expert Choice
8.Other(speciQ):
(6). Builds up statistical information about historical N k s that assist in better modeling of firture
project;
Used
(7). Leads to the use of the most suitable form of procurement/contract;
(8). Assists in distinpishing between good luck/good management and bad luclchad
management.
(9). Other reasons (specify):
8. If your cornpany does not use nsk analysis techniques, which of the following
describes the reason: (Circle one or more items)
(1). Waste of time and efforts to do risk analysis; (2). No tirne to do it;
(3). No money to do it;
(4). No intemal expertise to carry out risk analysis;
(5). No information available to cany out risk analysis;
(6). Other (speciQ):
9. Type and name of company you are in: (Circle one item)
(2). Engineering Firm
(3). Construction Contractor
(4). Other (speciQ)
Your company's name: (Please specifi Company and division)
10. Your curent position: (Circle one item)
(1 ). Project Estimator (2). Project Engineer (3). Project Manager
(4). Other (specifjQ
1 1. Years of your experience in Uidustry:
Thank you so much for your Pme, efforts, and insights.
2. Survey Results
(1). Project Size vs. Percentage of Responses:
S i t e of Project
- ~ M u s t Use
Q Better to Use
No Need to Use
(2). Software Packages Used vs. Frequency of Responses:
Softwate Packages Frequency of Responses
Crystal Ball(with Excel, Lotus) 16
@Ris k(for Excel) 21
Risk + (Wh MS Project) 26
@Risk for Project 11
Monte Cario (with PrÎmavera) 37
Expert Choice 11
Otherî (Dyadem Hazop)
D&RA
I n-iiouse Oevelo ped P rograms 26
REPlPC 32
(4). Benefits of Performing Risk Analysis vs. Frequency of Responses:
BENEFITS OF RISU ANALYSIS Frequency 01 Res~onses
Gives an increased understanding of the risks in a project
Allows the assessrnent of contingencies that actuaily reflect the 90 risks
Allows the formulation of more reaiistic plans, in terms of cost 71 estimate and timescale
Facilitates greater, but more rational, risk taking, thus increasing 58 the benefits that cm be gained fiom risk taking
L
Leads to the use of the most suitable fom of procurementkontract 29
Builds up statistical information about historical risks that assist in 26 better modeling of future project
Assists in distinguishing between good luck/good management and 26 bad lucklbad management
Identifies the party best able to handle a risk 13
APPENDIX II. CONSENT FORM
Consent for Industry Survey
Research Project Title: An Artificial Neural Network Approach to Assess Risk Effects and Project Decisions in the Front-End Stage
Researcher: Xiaoying Liu
Research Supervisor: Dr. George Sergeas
Funding: NSERClSSHRC
This consent form, a copy of which has been given to you, is only part of the process of infiormed consent. It should give you the basic idea of what the research is about and what your participation will involve. If you would like more detail about something mentioned here, or information not include here, please ask. Please take the time to read this form carefully and to understand any accompanying information.
n i e purpose of the study is to identiQ project cntical risk factors and to evaluate risk impacts on project performance, capital cost and schedule. You have been contacted because you have expenence in project fiont end planning and development. This consent form is to describe the industry survey to you and to request your consent to conduct one or more surveys.
The s w e y consists of a set of questions I would like to ask you. Al1 questions are related to a project you have worked on. One survey is to be completed for each project. The size of the reviewed project shouid be over S 1 MM in value and the project must have been completed within the past five years.
Participants participating through a personal interview will be asked to provide a time cornmitment of one to two hours. Interviews will be arranged at a time and place that is convenient for you.
Ni data colIected will be pooled in a database and no information fiom any specific individuai or Company will be released without pnor written authorization. The database
will be stored on diskettes and managed and kept directly by the researcher. Only the researcher's supervisor and the researcher have access to this information. You will have access to the information that you have provided. You will also have the oppomuiity to delete, change and destroy any information you have provided during the study. No othea will be allowed to use the coilected data without your permission. Aggregated (and de-sensitized) data that carmot be separated or identified with a particular contributor will be used to report research W i g s and will therefore be publicly accessible. If you decide to participate, you will need to carefully read and sign this consent form. Thank you for taking the tirne to read this information.
Your signature on this form indicates that you have understood to your satisfaction the information regarding participation in the research project and agree to participate as a subject. In no way does this waive your legal rights nor release the investigators, sponsors, or involved institutions fiom their legal and professional responsibilities. You are fiee to withdraw fiom the study at any t h e . Your continued participation should be as informed as your initial consent, so you should feel fiee to ask for clarification or new information throughout your participation. If you have questions conceming matters related to this research, please contact:
The Researcher: Xiaoyin~ Liu at 220-5970 or email: [email protected]
Researcher's Supervisor: Dr. George lergeas at 22041 85
If you have any questions conceming your participation in this project, you may also contact the Office of the Vice-President (Research) and ask for Karen McDermid, 220- 3381.
Signature of Participant Date
Signature of Researcher Date
APPENDIX III. SURVEY QUESTIONNAIRE
Industry Survey Oil and Gas Project9s Time and Cost
Overruns Predictive Mode1
Introduction
The purpose of this s w e y is to obtain project information on individual project characteristics, risks, decisions, and economic characteristics fiom the oil and gas industry. This idormation will be used in support of research being undertaken at The University of Calgary. The goal of this study is to develop an intelligent predictive model of project time and cost variances using artificial neural network technology.
The intelligent model will be used as a tool to evaiuate the effects of risks and project decisions on project outcornes. This will assist decision makers in improving the decision making processes at the fiont end stage of projects in the oil and gas industry.
Confidentiality
AN information obtained will be held in strict confidence by the researcher. AI1 data will be pooled in a database and no information fiom any specific individual or Company will be released without prior written authorization.
Guidance
Your answers to al1 questions should be related to a project you have worked on with a value over CD$lMM dollars and must have been completed within the past five years. One questiomaire form is to be completed for each project. There are four different sections in this form relating to: i Project details
Significant project factors affecting t h e and cost oveduder runs I d e n w the top ten significant attributes to time and cost variances Additional idormation you may wish to add
We would appreciate receiving completed foms for as many project as possible - the larger our database, the more useful the resuiting mode1 will be. We would also appreciate feedback on the most successful projects and the worst projects as measured by traditional project outcomes.
Data Accuracy
Al1 numenc answers need to be given a confidence Level (CL.). Please use the following scale:
Precise Approximate Guess
Note: 1. Precise: m e r based on accounting or other similar corporate data; 2. Approximate: Answer based on knowledge of situation, but no exact data
available; 3. Guess: Best esrimate based solely on experience W o r intuition.
************************************************************************
Questionnaire
Section 1 - Project details:
In this survey, the project's estimated cost refers to the estimated cost on which the initiation of the project detailed design was approved; the project's actual cost refers to the cost incurred when the project started to operate.
1. Estimated and actual cost:
(1.1). The project's estimated cost: ($Millions) . [C.L.: ]
Answer either (12) or (13).
(1.2). The project's actual cost: ($Millions) . [CL.: ]
or (1 3). The project actual cost as a percentage of the project's estimated coa (%): [CL.: 1
2. The contingency ailocated in the project's estimated cost: (%) . [C.L.: [
3. Estimated and actual duration:
(3.1). The project's estimated duration comsponding with the project's estimated cost(Months): [CL.: 1
Answer either (33) or (33).
(3.2). The actual duration of the project when the project started to operate(Months) . [C.L.: 1
or (3.3). The percentage of the project actual duration to the project's estimated duration corresponding with the project's estimated cos(%): [C.L.: 1
4. The contingency included in the project's estimated schedule identified in your answer to question 3.1.(%): . [C.L.: 1
5. In which year the project was commissioned: 199,
Section 2 -Project critical factors that have impacts on time and cost variances:
6. Type of the project: (Circle one number)
(1) Gas Plant (2) Pipeline (3) Refinery. (4) Offshore development
(5) Other, please specify
7. Project location: (Circle one number)
(1) Urban (2) Suburban (3) Rural (4) Remote
8. Project complexity:
(8.1). Technical cornpiexity of the project: (accessibility, technical requirements and limitations, number of rotating pieces of major equipment, specialty materials, degree of automation, equipment redundancy, ...) (circle one number)
(1) Low (2) Medium (3) Hi&
(8.2). Number of organizations directly participated in the project: [ CL.: 1 (Including Owners, Engineering Consultants, Contractors, Sub-Contractors, Suppliers)
9. The percentage of engineering design completeness to the entire project engineering design at the start of field construction: [C.L.: 1
10. Capitaüzed cost expended on the front end planning of the project(prior to detailed design)($): . [C.L.: 1
I l . The cost expended on detailed design of the project($): . [C.L.: 1
12. Rank the following project priorities for this project :
( ). Quality ( ). Safety ( ). Schedule ( ). Cost
13. Number of external permits required in total for the project: [C.L.: J (External perniîa including envimnmental, development, land use, construction, rond
use, ...)
14. The extent to which the project scope was firm and elearly defined when the initiation of the project detailed design was approved: (circle one nurnber) (Project scope and definition typically inciude generaf project basis, process design, site information)
-Project scope was vague; -~aven ' t documented project mission staternent; Poor Well
-Haven 't completed project pre-planning and site investigation. t
-Haven? done Design Basis 5 4 3 2 1
Memorandum & Engineering Design Specification
-Project scope was firm; -Have documented project mission statement; -Have completed project pre-planning and site investigation.
-Have done Design Basis Mernorandum & Engineering Design S pecification
15. Project management experience in engineering, procurement, construction and management (EPCIEPCM) projects when the project was proceeded:
(1 5.1). Your organizationldivision had project management experience in similar projects:
(Similar projects in ternis of similar type, similar technology used )
(1 5.2). Years of project management experience that the project management leader in the owner organization/division has had with EPCEPCM projects . [C.L.: 1
16. Contract strategy and type used on the project by the owner(s):
( 1 ) EPC or EPCM(Sing1e primary contractor)
Contract Type: (Circle one letter)
(a) Stipulated price (b) Cost plus (c) Unit price (d) other please speciQ:
(2) EPC or EPCM(Mu1tiple primary contractors)
Dominant Contract Type: (Circle one letter)
(a) Stipulated price (b) Cost plus (c) Unit price (d) other please specifjc
(3) Design-Bid-Build(0ne primary contractor)
Contract Type: (Circle one letter)
(a) Stipulated price (b) Cost plus (c) Unit price (d) other please specifi:
(4) Design-Bid-Build( Multiple primary contractors)
Dominant Contnrct Type: (Circle one letter)
(a) Stipulated price (b) Cost plus (c) Unit pnce (d) other please specify:
(5) Partnering or alliance
(Note: Design-Bid-Build means project engineering is separated with procurement and construction)
17. The level of technological innovation of the project: (Circle one nurnber)
(1) Established, very familiar to your organization;
(2) Established in the industry but new to you organization;
(3) New technology to the industry or sector but established and used in other industry or sector;
(4) Brand new technology application to any indusüy.
18. The percentage of construction actMties adverseiy affected by weather during the project constniction phase(%): [C.L.: 1
Section 3 -4dentify the top ten significant factors to cost and time variances from Section 2.
Section 4 - Additional Information:
Please list additional factors you think that are criticai to the project:
1.
Please indicate the category which best describes your current position:
Senior Executive(VPICE0) Project Engineer
Senior Project Manager Project Manager
- Others, please specQ
Wouid you like to receive a bnef copy of the results? Yes No -
Your narne:
Mailing Address:
Thank you so much for your time, efforts, and insights.
APPENDIX IV. DEFINITION OF VARIABLES
Eighteen variables and thek descriptions used in this study are listed as the following:
'Dumrny' variables: 1 : Project is a gas plant. V12: Project is a pipeline. V 1-3 : Project is a rehery. V14: Project is an offshore Development. V15: Others
V2: Project Location
'Dumrny' Variables: V2-1: Project is located in an urban area. V2-2: Project is located in a suburban area. V2-3: Project is located in a rurai area. V2-4: Project is located in a remote area.
V3: Project Complexity
'Dummy' Variables: V3-1: Project complexity is low. V32: Project complexity is medium. V33: Project complexity is high.
V4: The number of key organizations directly involved in the project
'Dumrny' Variables: V4-1: The number of key organizations is less thankqual to 5. V4-2: The number of key organizations is greater than 5 but less thanlequa1 to
15. V4-3: The number of key organizations is greater than 15 but less thdequa1 to
30. V4-4: The number of key organi;rations is greater than 30 but less than/equal to
50.
V4-5: The number of key organizations is greater than 50.
V5: The percentage of engineering design completeness to the entire project engineering design at the start of field construction
V6: The cost expended on the front end planning of the project
V7: The cost expended on the detailed engineering of the project
V8: Project priori@ on quality
V9: Project priority on safety
V10: Project priority on schedule
V 1 1 : Project priority on cost
V12: The numbea of extemal permits/Approvals required in total for the project
'Dummy' Variables: V12-1: The number of permits/approvals is less thadequal to 5. V 12-2: The number of perrnits/approvais is greater than 5 but less
thdequai to 10. V12-3: The number of permitslapprovals is greater than 10 but less
thadequal to 1 5. V12-4: The number of permits/approvals is greater than 15 but less
thanfequal to 20. V12-5: The number of permits/approvals is greater than 20.
V13: The extent to which the project scope was defined when starting detailed engineering
V 14: Having experience in similar projects
'Dummy' Variables: V14-1: The project team has experience in sllnilar projects. Vl4-2: The project team has no experience in similar projects.
VI 5 : Years of experience that the project management leader(s) has had with projects
V16: Type of contract
'Dummy' Variables: Vl6-1: Contract type is stipulated price. V 16-2: Contract type is cost plus. Vl6-3: Contract type is unit pnce. V 16-4: Contract type is partnering or alliance. Vl6-5: Others.
V 17: Level of technological innovation of the project
' Dummy ' Variables: V17-1: The technology used in the project is established, very familiar to
your organization. V 17-2: The technology used in the project is established in the industry
but new to your organization. Vl7-3: The technology used in the project is new technology to the
industry/sector but established and used in other industry/sector. V 17-4: The technology used in the project is brand new technology
application to any indu-.
V 18: The percentage of construction activities af5ected by weather
'Durnmy' Variables: V18-1: The percentage of construction activities affected by weather is
less thadequal to 5%. V18-2: The percentage of construction activities affected by weather is
greater than 5% but less thadequal to 25%. VI-: The percentage of construction activities affected by weather is
greater than 25%.
APPENDIX V. A SAMPLE OF A BATCH FILE FOR ANN TRAINING
An exarnple of a batch file for ANN training is listed below:
% "pipeline projects: total 36 samples. 32 samples for training 4 for testing. % 90% training sets=32; testing sew; Logsig-pureline; % Input variables: 10(V 1 ,V2,V3 ,V4,VSYV6,V7,V 1 1 ,V 1 5,V 1 8);
% The foîiowings are for calcdating Standard Deviation. C=pred-T-T-test; C l=C1; STD-test=std(C 1); D=pred-'IT-T; D 1 =Dl; STDTDtrain=std(D 1);
% The foiiowings are for calculating R squre(corre1ation coefficient). %training set: for i=I:32
cost-train(i)=T(l ,i);
costpred(i)=pred-TT(1 ,i); tirne_train(i)=T(2, i); timegred(i)=pred_TT(2,i);
end Rcost~train=co~coef(cost~train~costpred); Rtime-train=corrcoe f(time-train,timem;
%Testhg set: for i=1:4
cost-t est (i)=T-test ( i,i); costpredT(i)=pred-T(1 ,i); time_test(i)=T_test(2,i); &nejredT(i)=pred-T(2,i);
end Rcost~test=corrcoef(cost~test,cost~redT); Rtime~test=corrcoef(time_test,tirne~redT);
save out corn STD-test STD-train Rcost-train Rcost-test Rtimctest
l MAGE EVALUATIO N TEST TARGET (QA-3)
APPLIED IWGE . lnc 1653 East Main Street -
=A Rochester. NY 14609 USA ,=-A Phone: 71 6/4l8~93oO -- - Faxr 716t288-5989
