Safety Science 47 (2009) 425–435

Contents lists available at ScienceDirect

Safety Science

journal homepage: www.elsevier .com/locate /ssc i

A real-time warning model for teamwork performance and system safetyin nuclear power plants

Sheue-Ling Hwang a,*, Guo-Feng Liang a, Jhih-Tsong Lin a, Yi-Jan Yau a, Tzu-Chung Yenn b,1,Chong-Cheng Hsu b,1, Chang-Fu Chuang c,2

a Department of Industrial Engineering and Engineering Management, National Tsing Hua University, 101, Section 2, Kuang-Fu Road, Hsinchu 300, Taiwanb Institute of Nuclear Energy Research, Atomic Energy Council, Executive Yuan No. 1000, Wunhua Road, Jiaan Village, Longtan Township, Taoyuan County 325, Taiwanc Atomic Energy Council, 6F, No. 80, Section 1, Cheng Kung Road, Yung-Ho City, Taipei Country 234, Taiwan

a r t i c l e i n f o a b s t r a c t

Article history:Received 6 November 2007Received in revised form 26 April 2008Accepted 14 July 2008

Keywords:Real-time warning model (RTWM)TeamworkNPPsMental WorkloadGMDHFuzzy logic

0925-7535/$ - see front matter � 2008 Elsevier Ltd. Adoi:10.1016/j.ssci.2008.07.011

* Corresponding author. Tel.: +886 3 5742694; fax:E-mail addresses: slhwang@ie.nthu.edu.tw (S.-L.

edu.tw (G.-F. Liang), d947808@oz.nthu.edu.tw (J.-T. L(Y.-J. Yau), tcyenn@iner.gov.tw (T.-C. Yenn), worchuang@aec.gov.tw (C.-F. Chuang).

1 Tel.: +886 3 4711400; fax: +886 3 4712358.2 Tel.: +886 2 22322118; fax: +886 2 22322113.

In order to increase system safety and team performance, this study aimed to develop a real-time warn-ing model (RTWM) by assessing team response time, error rates, and mental workload. Toward this goal,the group method of data handling (GMDH) algorithm was applied to physiological indices to predictteam performance. Then fuzzy logic, fuzzy inference and linguistic variable sets representing the TeamPerformance and Safety Index were applied to construct the RTWM. To model the RTWM, experimentswere conducted on computer-supported cooperative work (CSCW) in the personal computer transientanalyzer (PCTRAN) simulator. The simulator and teamwork are designed to simulate the real tasks ofthe control room of the Fourth Nuclear Power Plant (FNPP) in Taiwan. In addition, important physiolog-ical parameters, the NASA-TLX questionnaire, team response time, and team error rates were collectedfrom 39 participants. The results revealed that there was a significant positive correlation between theerror rates of teamwork and the interval of event arrival time. This indicated that a pre-alarm device isnecessary because vigilance decreased with time. Moreover, a predictive teamwork performance modelapplying the GMDH algorithm and the RTWM with a fuzzy inference system was developed in this study.The proposed model can efficiently predict teamwork performance to maintain appropriate mental work-load as well as ensure system safety.

� 2008 Elsevier Ltd. All rights reserved.

1. Introduction

Recently, there has been increased interest in the study of teamcollaboration. In particular, one of the most popular topic areas inteamwork research has focused on computer-supported coopera-tive work (CSCW) which provides an automated system and sharedenvironment to support teamwork. Team members have to moni-tor tasks and share responsibility. However, some key problems insuch an environment need to be resolved, such as (1) how to main-tain high team performance, and (2) how to ensure human andsystem safety when sharing responsibility in monitoring tasks.

Numerous studies have noted that team performance measure-ment is more complicated than that of individual performance be-cause teamwork relies heavily on communication, supervision of acommon situation, and the sharing of the mental workload (Carv-

ll rights reserved.

+886 3 5722685.Hwang), d947803@oz.nthu.

in), d927814@oz.nthu.edu.twm@iner.gov.tw (C.-C. Hsu),

alho and Vidal, 2007; Sebok, 2000). Various methods can be takento measure team performance, and most of these methods focus onhuman reliability analysis (HRA), cognition, team interactions, andobjective performance (Artman, 2000; Sebok, 2000; Shu andFuruta, 2005; Marseguerra and Zio, 2006).

In addition, some studies have revealed that mental workload isrelated to human performance (Xie and Salvendy, 2000; DiDome-nico and Nussbaum, 2005; Desmond and Hoyes, 1996). Hollnagel(2003) indicated that cognitive workload is regarded as that subsetof mental workload which requires conscious effort (e.g., recogniz-ing familiar objects or driving a car). Moray (1988) suggested thatthe appropriate mental workload could reduce human errors andenhance system safety. Rubio and Diaz (2004) pointed out thatproper mental workload ensured the safety and long-term produc-tion efficiency of operators. Greef et al. (2007) reviewed the symbi-otic relation between man and machine which aimed tocompensate for temporal limitations in human information pro-cessing, overload, cognitive lockup, and underload by a combina-tion of performance, effort, and task information.

In the nuclear power plant control room, team members havebeen assigned roles, responsibility, and areas of specialization butwere often cross-trained to be able to deal with operating and

426 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435

monitoring tasks. The sharing of responsibility, also termed diffu-sion of responsibility, with a computer for system monitoringand decision making tasks may achieve shared team workloadand shared team mental models (Skitka et al., 2000; Rentsch andKlimoski, 2001). However, diffusion of responsibility may reducethe effort and workload of group members who have capable part-ners free-riding on their efforts (Mclntyre et al., 2003). For suchteam members with diffusion of responsibility, the underload ofeach team member may influence the overall team performanceespecially in monitoring tasks.

Several methods have been developed to measure mental work-load. These methods are classified into three categories: perfor-mance-based measures, subjective measures (e.g., NASA-TaskLoad Index), and physiological measures (e.g., eye blink, respira-tion, electroencephalogram (EEG), heart rate (HR), heart rate vari-ability (HRV)) (Luximon and Goonetilleke, 2001; Braarud, 2001;Laine and Bauer, 2002; Wilson and Russell, 2003; Rubio and Diaz,2004). Among these methods, heart rate variability can provide theclearest evidence and can provide real-time support for the use ofphysiological activity in a test (Kamada et al., 1992a,b; Kageyamaand Kabuto, 1995; Nishikido et al., 1997; Scerbo and Freeman,2001; Mikulka et al., 2002; Stanton et al., 2005). Tattersal andHockey (1995) showed that the heart rate increased and heart ratevariability spectrum (HRVs) decreased in flight engineers whengiven three levels of cognitive task demands during the flight tasks(i.e., system monitoring, routine fault correction, and problem solv-ing). In addition, Sammer (1998) compared a physical task, a cog-nitive task, and a combination of both task with computing theheart period (IBI) and heart rate variability (HRV) in the low(0.01–0.05 Hz), mid (0.06–0.16 Hz), and high (0.2–0.4 Hz) range.The results showed that the HRV was less for the dual task andgreater for the physical and cognitive tasks. Scerbo and Freeman(2001) reported that the heart rate (HR) increases and heart ratevariability (HRV) decreases with increased mental workload, andone of the advantages with using HRV as a measure is the capabil-ity to have continuous, on-line recordings. Therefore, heart ratevariability is suitable for constructing a real-time predictive modelto ensure both system and human safety.

Safety is an important criterion during design and management(Hale et al., 1997, 2007; Schupp et al., 2006). According to previousstudies, team model, team performance measure methods, andmental workload are proposed to ensure safety; however, these re-searches have focused on concepts, and very few studies have beenconducted on the predicting performance and deciding the safetythreshold in practical work. The purpose of this study was (1) todesign a predictive teamwork performance model using the groupmethod of data handling (GMDH) algorithm, and (2) to determinethe safety threshold by fuzzy logic when team members wereunderloaded mentally.

The group method of data handling (GMDH) algorithm has beenwidely applied in various fields (e.g., education, economic systems,weather modeling, manufacturing, pattern recognition, physiolog-ical experiment (Baker and Richards, 1999; Pavel and Miroslav,2003; Sarycheva, 2003; Kim et al., 2001; Ivakhnenko, 1993, 1995;Hwang et al., 2008)). This paper extended the GMDH algorithm todevelop the proposed team performance predictive model.

Fuzzy logic is based on the fuzzy set theory which provides amethodology that simulates human thinking by explicitly model-ing and managing the linguistic imprecision and uncertain (Zadeh,1965). Currently, it has been used for modeling in many fieldsincluding workload assessment, intelligent patient monitoringand alarm systems, human–machine systems, cognitive workload,electrocardiograms and electroencephalogram signal design, andautomated assistance system (Becker et al., 1997; Ntuen, 1999;Moon et al., 2002; Liu and Su, 2006; Gregoriades and Sutcliffe,2006; Rani et al., 2007; Yang et al., 2008). In this study, fuzzy logic

was used to determine the linguistic index, performance, and thendefine the linguistic threshold and Safety Index to avoid underloadfrom the diffusion of responsibility in monitoring tasks.

Therefore, firstly the methodology of GMDH and fuzzy logic aredescribed in Sections 2 and 3. The details of the experimentalmethods and design are presented in Section 4. The team perfor-mance prediction model and the threshold of Safety Index usingGMDH and fuzzy logic are described in Section 5. Section 6 dis-cusses the findings and limitations of the proposed models inteamwork. Finally, Section 7 provides some comments about thisstudy.

2. The group method of data handling (GMDH)

The GMDH algorithm, a self-organizing approach, was a pio-neering proposition by Ivakhnenko (1968) for identifying the bestpredicting polynomial equation. The algorithm found the onlyoptimal model by fully sorting out model-candidates and evaluat-ing their operation by using external criteria of accuracy or differ-ence types (Madala and Ivakhnenko, 1994). Recently, geneticalgorithms and neural network methods have been combined intoGMDH polynomials, and some software packages have been devel-oped by the Glushkov Institute of Cybernetics and Algorithm forSynthesis of Polynomial Networks (ASPN) such as NeueroShell2and ModelQuest (WSG, 1995).

GMDH nets derive a mathematical formula which is a nonlinearpolynomial expression relating the values of the most importantinputs to predict the output variable. The general connection be-tween input and output variables can be expressed by the Volterrafunctional series, the discrete analogue of which is the Kolmogo-rov–Gabor polynomial (Madala and Ivakhnenko, 1994):

y ¼ a0 þXM

i¼1

aixi þXM

i¼1

XM

j¼1

aijxixj þXM

i¼1

XM

j¼1

XM

k¼1

aijkxixjxk ð1Þ

where X(x1,x2, ... ,xM) is the vector of inputs and A(a1,a2, ... ,aM) is thevector of coefficients or weights. The components of the input vec-tor X can be independent variables, functional forms, or finite differ-ence terms. Other non-linear reference functions such as difference,logistic, and harmonic can be used for model construction.

In this study, a team performance prediction model using phys-iological data as the input variable and using the NeueroShell2 toolwas established. Furthermore, the output values of this modelwere transferred into the input values of the fuzzy interferencesystem as described in the next section.

3. Fuzzy Logic

Zadeh introduced the term fuzzy logic (FL) describing the math-ematics of the fuzzy set theory in 1965. Fuzzy logic provides theability to mimic the human mind to employ modes of reasoningthat are approximate rather than exact. It can also address linguis-tic imprecision and tolerance. A rule-based system, the fuzzy infer-ence system (FIS) is one of the most famous applications of fuzzylogic and the fuzzy sets theory. In this section, we shall brieflyintroduce the fuzzy set theory and fuzzy inference system whichis applied in this paper.

3.1. Fuzzy sets and membership functions

The classical set theory has a crisp definition on whether an ele-ment is a member of a set or not. However, a fuzzy set is an exten-sion of a crisp set. Crisp sets allow only full membership or nonmembership, whereas fuzzy sets allow partial membership inwhich a degree of membership ranging between zero and one is as-signed to each element. A fuzzy set A, symbol ~A; on a universe U is

Fuzzification Inference

engine Defuzzification

Rule base

Agregation

XY

Fig. 1. Fuzzy inference system.

S.-L. Hwang et al. / Safety Science 47 (2009) 425–435 427

characterized by a membership function l~AðxÞ. Various types ofmembership function are used, including the triangular type, trap-ezoidal type, generalized bell shape, Gaussian curves, polynomialcurves, and sigmoid function (Klir and Yuan, 1995).

3.2. Fuzzy inference system

A fuzzy inference system (FIS) uses the fuzzy rule in a nonlinearmapping of input linguistic variables into a scalar output. A modelof FIS contains five stages which are as follows (see Fig. 1): (1) fuzz-ification, (2) inference engine, (3) rule base, (4) aggregation, and (5)defuzzification (Mamdami and Assilina, 1975; Kruse et al., 1994;Kulkarni, 2001).

Fuzzification is a process which takes input values and deter-mines the degree to which they belong to each of the fuzzy setsvia membership functions. The inference engine determines thedegree to which the antecedent is satisfied for each rule. The rulesare obtained from expert knowledge, sample data points, or train-ing data (Wang and Mendel, 1992; Dagdeviren et al., 2008). In or-der to get a crisp value for the output, we needed a defuzzificationprocess. Many defuzzification techniques are being proposed invarious studies including methods of center of gravity (COG), max-imum decomposition, center of maximum, and the height method.The most commonly used method is the center of gravity, whichdetermines the center of an area and uses this value as the output.

To model the fuzzy inference system, the input linguistic vari-able and teamwork performance should be obtained. The team-work performance can be predicted by the physiological datafrom this study. Thus, an experiment was conducted to collectthe physiological data and teamwork performance data.

ProcedBP1~2

Fig. 2. Power generator co

4. Method

4.1. Participants

Forty-five participants in fifteen teams took part in the experi-ment. They included 28 students (average age = 26.68 years;SD = 4.52 years) of the National Tsing Hua University, 14 experts(average age = 45.86 years; SD = 12.46 years) with an average of19.64 working years in the Institute of Nuclear Energy Research,and three operators (average age = 39 years; SD = 1.73 years) withan average of 13.67 working years at the Fourth Nuclear PowerPlant in Taiwan. Before running the experiment, six students intwo teams joined a preliminary experiment. The remaining 39 par-ticipants attended the formal experiment.

4.2. Apparatus

The Cardio Tens portable device made by Meditech was used anelectrocardiogram (ECG). The ECG signals were calculated usingthe software from cardio visions in order to transfer the signalsinto heart rate variability (HRV) indices including time domainand frequency domain. The personal computer transient analyzer(PCTRAN) system including the power generator control system(PGCS), the reactor recirculation system (RCIR), and the rod controland information system (RCIS) were used to simulate the startupreactor task. A program in the training simulator was designed torepresent the procedures of integrated operating procedure (IOP)201.2, reactor startup with PGCS, and 202.2, power changes withPGCS. In addition, operators executed the tasks by mouse followingthe operating procedure.

Continuous Bottom

ure: 9

ntrol system (PGCS).

Core

Vessel

Dryer

Wetwell

Fig. 3. Reactor parameters display.

428 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435

4.3. Experimental tasks and variables

The team members organized by the reactor operator (RO),assistant reactor operator (ARO), and supervisor were given differ-ent tasks as part of the teamwork. The primary task of the team-work was to start up the reactor safely in the PCTRAN systemprovided by the Institute of Nuclear Energy Research (INER). More-over, the seven setting points and 29 procedures from break-point1 (BP1) to break-point 29 (BP29) were established in the PGCS ofPCTRAN. In the teamwork, the tasks of the reactor operator wereto continuous push the button in PGCS (see Fig. 2) in order to carryout 29 procedures, and input the target value of the core flow andpower value at each setting point by the standard operation proce-dure (SOP); meanwhile, the tasks of the assistant reactor operatorwere to monitor the parameters of the core, the vessel, dryer, andwetwell, and to judge if the parameters were under control (seeFig. 3). Afterwards, the assistant reactor operator wrote down theparameters of the water level, temperature, pressure, core flow,and power value in the standard operation procedure (SOP) and re-layed these values to the supervisor when the core flow or power

PGCS interface

RCIR interface

RCIS interface

Core flow and Power interface

Tarreq

Fig. 4. Interface layou

value had reached each setting point. Similarly, the supervisorsupervises the reactor operator and assistant reactor operator inperforming the procedures correctly as stated in the standard oper-ation procedure (SOP). The supervisor recorded all of these valuesat the same time.

The team members’ position and the interface layout in thisexperiment are shown in Fig. 4. The participants in this experimentas shown from left to right were the reactor operator, the supervi-sor, and the assistant reactor operator, respectively. The devicesusing in this experiment consisted of seven monitors, two comput-ers, two mice, and two keyboards.

The independent variable in this experiment was the interval ofevent arrival time, and the dependent variables were physiologicalindices (HRV indices), teamwork performance (response time andcorrect rates), and mental workload which are described asfollows.

The PCTRAN system provided the system default value andinterval time; for example, (1)–(2) means the interval of the arrivaltime between event 1 and event 2 in the scenario.

Turbine system interface

Alarm priority list interface

get value uiring interface

Alarm items interface

ts of teamwork.

S.-L. Hwang et al. / Safety Science 47 (2009) 425–435 429

The heart rate variability (HRV) indices were calculated fromshort-term (5 min) and long-term (24 h) recordings from the elec-trocardiogram (ECG) such as time-domain and frequency-domainanalysis presented as follows (McNames et al., 2003):

(1) Time-domain metrics(1) NN number (Num): number of normal-to-normal (NN)

intervals.(2) NN average (ms): average on NN intervals in ms.(3) SDNN (ms): the standard deviation of the NN intervals.(4) pNN50 (%): number of NN interval pairs with over

50 m s difference.(5) HRVti: HRV triangular index which is a measure of the

shape of the NN interval distribution. Uniform distribu-tions representing large variability have large valuesand distributions, while single large peaks have smallvalues.

(6) RMSSD: the root mean square of successive NN intervaldifferences.

(2) Frequency-domain metrics(1) LF: the low frequency (LF) power which was calculated

as the total signal power in the frequency range of0.04–0.15 Hz.

(2) HF: the high frequency (HF) power which was calcu-lated as the total signal power in the frequency rangeof 0.15–0.40 Hz.

(3) LF/HF: the low frequency–high frequency ratio.(4) TP: the total power which was the integral of the power

spectral density (PSD) estimated over the full frequencyrange of 0.0–1.5 Hz.

In this experiment, the teamwork performance of the primarytask provided data of error rates (or correct rates) and the responsetime of the teams. Furthermore, the subjective mental workloadwas evaluated using the questionnaire NASA task load index(TLX) at the end of the experiment.

4.4. Experimental procedure

There were several stages before the experiment. First, eachparticipant wore the Cardio Tens portable device and to checkwhether the electrode was regular or not. The default settings onthe device began to collect the heart rate variability data after10 min of wearing the apparatus. The experimental tasks for theteamwork were enumerated, and each participant was guided inreading the standard operation procedure (SOP) and took theNASA-TLX questionnaire. They were asked to sign an informedconsent once the procedure was explained to them. Afterwards,the initial heart rate variability (HRV) indices were automaticallymeasured around 5 min as a base line before the experiment. Atthe beginning of the experiment, each team was given 10 min tofamiliarize themselves and to practice the control procedureaccording to the standard operation procedure (SOP) and theinstructions. Then the team members implemented and monitoredthe startup reactor including tasks such as communicating, input-ting the target value, recording the vital parameters, and so on.Moreover, the HRV indices were recorded continuously by the de-vice until the startup reactor finished which was after approxi-mately 40 min. Finally, the NASA-TLX questionnaire was filledout by the team member to evaluate the subjective mental work-load after the experiment.

5. Results

In developing a real-time warning model (RTWM) firstly, in Sec-tion 5.1 we surveyed the team members’ mental workload in the

monitoring tasks of this experiment. In addition, the correlationbetween the interval of event arrival time and the error rates ofthe 13 teams were analyzed in Section 5.2. Then, the GMDH withthe neruoshell2 tool and fuzzy logic theory were applied. The pro-cess of developing the RTWM contains two models as shown inSections 5.3 and 5.4. Finally, the two models were validated in Sec-tion 5.5.

5.1. Subjective questionnaire assessment of mental workload

The NASA-TLX was used as a subjective workload measure. TheTLX has six subscales: mental demand, physical demand, temporaldemand, performance, effort, and frustration. To investigate themental workload of teamwork undertaken in this experiment,the questionnaire integrated NASA-TLX with five-point Likertscales (strongly low, low, middle, high, and strongly high) and fuz-zy calculation. The fuzzy scale of each band of the five-point scalewas calculated from the responses of 39 participants in 13 teams.Each team included a reactor operator (RO), an assistant reactoroperator (ARO), and a supervisor. The mental workload scores(mean and standard deviation) of the reactor operator, the assis-tant reactor operator, and the supervisor were calculated aswR = 37.96 ± 14.29, wA = 39.37 ± 10.08, and wS = 35.65 ± 16.46,respectively. The fuzzy membership was obtained as follows:

RO : fwR;liðwRÞg ¼fðl2ðwRÞ;0:6Þ; ðl3ðwRÞ; 0:66Þ;ðl4ðwRÞ; 0:13Þg ð2Þ

ARO : fwA;liðwAÞg ¼fðl2ðwAÞ;0:52Þ; ðl3ðwAÞ;0:7Þ;ðl4ðwAÞ; 0:16Þg ð3Þ

Supervisor : fwS;liðwSÞg ¼ fðl2ðwSÞ;0:71Þ; ðl3ðwSÞ; 0:58Þ;ðl4ðwSÞ;0:02Þg ð4Þ

where the fuzzy membership of wR, wA and wS is represented asli(wR), li(wA),and li(wS), i = 1, 2, 3, 4, and 5 which means the degreeof membership of strongly low, low, middle, high, and stronglyhigh.

In Eqs. (2)–(4) the mental workload of the reactor operator,assistant reactor operator, and supervisor belongs to low and mid-dle in the PGCS startup reactor and monitoring tasks. This impliesthat all participants engaged in this semi-automated teamworktask perceived a low to middle mental workload.

5.2. Correlation between the interval of event arrival time and errorrates

The person correlation analysis was used to examine the rela-tionship between the interval of the event arrival time and errorrates of the 13 teams using statistical products and services solu-tion (SPSS). Table 1 indicated that the interval of event arrival timeand error rates were positively correlated with each other. The cor-relation coefficient of 0.966 was found to be statistically significant(p < 0.01, two-tailed). In other words, the longer the interval ofevent arrival time, the higher the error rates of the 13 teams(Fig. 5). Therefore, a teamwork performance prediction modeland warning model are deemed necessary to increase the vigilanceof team members especially during a longer event arrival time.

5.3. Teamwork performance prediction model

The first model was used to establish the relationship betweenthe physiological indices and correction rate/response time (C/R ra-tio). Thirty sets of data from 10 teams were used to construct themodel while nine sets of data from three teams were used to val-idate this model using NeuroShell software 2 with the GMDH algo-rithm method. The predictors of the eight heart rate variability

Table 1Pearson Correlation analysis between error rates and interval of event arrival time

Correlations Interval of the eventarrival time

Error rates madeof 13 teams

Interval of the eventarrival time

PearsonCorrelation

1 0.966a

Significance(two-tailed)

– 0.001

N 6 6

Error rates made of13 teams

PearsonCorrelation

0.966a 1

Significance(two-tailed)

0.001 –

N 6 6

a Correlation is significant at the 0.01 level (two-tailed).

0.00

0.05

0.10

0.15

0.20

0.25

0 50 100 150 200Interval of event arrival time (seconds)

Err

or r

ates

mad

e of

13

team

s (%

)

Error rates (%)(1)-(2)

(2)-(3)

(3)-(4)(4)-(5)

(5)-(6)

(6)-(7)

Fig. 5. Relationship between interval of the event arrival time and error rates.

0

0.2

0.4

0.6

0.8

1

Y1 Y2 Y3 Y4 Y5 Y6 Y7

C/R Rate

valu

e

R squared

Corr. coeff.

Fig. 6. Sensitivity test in performance index.

430 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435

(HRV) indices collected from the Cardio Tens as physiological indi-ces including NN count (X1), NN average (X2), SDNN (X3), pNN50(X4), HRVti (X5), RMSSD (X6), LF/HF (X7), and TP (X8) were adoptedto predict the performance index from the algorithm. To reduceindividual differences and eliminate the unit from each index, allof the predictors (Xi) were transferred from:

ðXiA � XiBÞ=XiA ¼ Xi ð5Þ

where XiA is the value during the experiment and XiB is the value be-fore the experiment. By Eq. (5), the range of the Xi scale is from �1to 1. In addition, the performance index (Y, %) is combined with thecorrect rate (y1) and the response time (y2) of teamwork at the sametime, which can be represented as

½ðy1Þ3=ffiffiffiffiffiy2p � � 10 ¼ Y ð6Þ

where (y1)3 andffiffiffiffiffiy2p

are simulated from the GMDH algorithm usingthe neruoshell2 tool. The concept of simulation is similar to sensitiv-ity analysis which investigates the effect on the optimal solution ifthe parameters take on other possible values. The results in Table 2reveal that taking the correct rate on (y1)3 and a fixed response time

Table 2Sensitivity analysis

Correct rate Response time C/R

y1ffiffiffiffiffiy2p

y1=ffiffiffiffiffiy2p ¼ Y1

y21

ffiffiffiffiffiy2p

y21=

ffiffiffiffiffiy2p ¼ Y2

y31

ffiffiffiffiffiy2p

y31=

ffiffiffiffiffiy2p ¼ Y3

y41

ffiffiffiffiffiy2p

y41=

ffiffiffiffiffiy2p ¼ Y4

y51

ffiffiffiffiffiy2p

y51=

ffiffiffiffiffiy2p ¼ Y5

y61

ffiffiffiffiffiy2p

y61=

ffiffiffiffiffiy2p ¼ Y6

y71

ffiffiffiffiffiy2p

y71=

ffiffiffiffiffiy2p ¼ Y7

onffiffiffiffiffiy2p

has the highest correlation coefficient (0.91) and R squarevalue (0.82), and all individual variables except for the standarddeviation of the normal-to-normal intervals (SDNN) are significant.Furthermore, the parameter variation values of the predictivemodel were scattered in Fig. 6 where the performance index Y3 isbetter than the others. Thus, given the values for X1, X2, X4, X5, X6,X7, and X8, a work performance predictive model can be expressedby

Y ¼ 0:68þ 0:99X5 þ 0:24X8 � 4:7X2 þ 0:67X4 � 19X21

� 0:15X27 � 0:51X3

4 � 5:1X1X4 � 6:6X1X7 þ 0:47X4X7

þ 15X1X4X7 þ 5X26 þ 2:20:002X3

2 � 8:9X36 � 2:6X2

5 � 1:1X28

þ 3:4X5X8 � 8X22 ð7Þ

To diagnose the team Safety Index, the prediction performancevalue of team members found using Eq. (7) could be used as an in-put into the fuzzy inference system.

5.4. Teamwork real-time warning model (RTWM)

The purpose of the second model is to maintain high team per-formance using a RTWM. To construct the RTWM, five steps couldbe conducted as follows:

Step 1: Draw the correct rate/response time (C/R) value to realizethe data scatter. From the practical value of this experi-ment, the performance index of C/R in 10 teams weresorted from low to high as shown in Fig. 7 whereas therest of the three teams remained to validate the model.The meaning of each dot represents the number of partic-ipants whose C/R was the same value.

Step 2: Determine the membership function and fuzzy numbersof performance indices. According to the scatters inFig. 7, three fuzzy numbers and membership function(l~jðYiÞ; i = R, A, S; j = low, middle, high) were modeled asEqs. (8)–(10).

R squared Correlation coefficient Insignificant

0.77 0.88 rMSSD0.80 0.89 HRVti0.82 0.91 SDNN0.67 0.82 rMSSD0.74 0.86 rMSSD0.73 0.85 SDNN0.74 0.86 SDNN

0

1

2

3

40.

010.

030.

050.

070.

090.

110.

130.

150.

170.

190.

210.

230.

250.

270.

290.

310.

330.

350.

370.

390.

410.

430.

450.

470.

490.

510.

530.

550.

570.

590.

610.

630.

650.

670.

690.

710.

730.

750.

770.

790.

810.

830.

850.

870.

890.

910.

930.

950.

970.

99

C/R value

num

ber

of p

artic

ipan

ts

number of participants

Fig. 7. C/R value of team members.

S.-L. Hwang et al. / Safety Science 47 (2009) 425–435 431

8

Fig. 8. Example of fuzzy rule bases.

llowðYiÞ ¼1; Yi < 0:26ð0:5� YiÞ=0:24; 0:26 6 Yi < 0:50; Yi P 0:5

><>: ð8Þ

lmiddleðYiÞ ¼0; Yi < 0:26 or Yi P 0:69ðYi � 0:26Þ=0:24; 0:26 6 Yi < 0:5ð0:69� YiÞ=0:19; 0:5 6 Yi < 0:69

8><>: ð9Þ

lhighðYiÞ ¼0; Yi < 0:5ðYi � 0:5Þ=0:19; 0:5 6 Yi < 0:691; Yi P 0:69

8><>: ð10Þ

where l~jðYiÞ is the membership function that the C/R ratioof the reactor operator, the assistant reactor operator andthe supervisor belong to.

Step 3: Transfer the linguistic variable, which is the performanceindex of the team members, as the input variable of thefuzzy inference system from the crisp value into the fuzzyset a process called fuzzification. To validate the proposedsystem, the data from the 10 teams was adopted in ran-dom, and then the rest of the three teams were retained.For instance, in the first team, the C/R ratio of RO was 0.34,implying that the degree of membership wasllow(0.34) = 0.66 and lmiddle(0.34) = 0.32. Similarly, theC/R ratio of ARO is 0.54 lmiddle(0.54) = 0.79 andlhigh(0.54) = 0.21), and that of the supervisor was 0.34(llow(0.34) = 0.66 and lmiddle(0.34) = 0.32). The calcula-tion for the rest of the nine teams is the same as this step.

Step 4: Construct the inference engine and rule base. The infer-ence engine determines the degree to which the anteced-ent is satisfied for each rule. The rule base is derived fromthe combination of fuzzy sets and the fuzzy numbers oflj(Yi), where i = R, A, S; j = low, middle, high, which has3 � 3 � 3 = 27 different combinations. For example, inthe first team, the membership value was 0.66 for theRO or the supervisor with low performance, and 0.32 withmiddle performance. Similarly, the membership valuewas 0.79 for the ARO with middle performance, and0.21 with high performance. The rule base in this casehad 2 � 2 � 2 = 8 combinations as shown in Fig. 8.

In Fig. 8, the fuzzy set output is defined as the linguistic notion,the team safety performance, described by three attributes: safety(Green signal), attention (Blue signal), and danger (Red signal). LetSI (Safety Index) represent the degree of team safety performanceand the range be [0,10]. If SI is lower, it means that the team per-formance is poorer and the warning condition is more critical. Toensure that all of the team members are in suitable performancelevel, when the premise of each team member falls into the lowthreshold, the conclusion would be in the Red of the SI0c where cis 1, 2, . . . ,r and r is the number of the rules.

Step 5: Execute the aggregation. According to the inferenceengine and rule base in the first team, the firing strength(ai) with an implication process of eight antecedents andconclusion is calculated as shown in Fig. 9. Then, theoutput of the aggregation process becomes the com-bined output fuzzy set by lSIðYÞ ¼max½l01lðYÞ;l02lðYÞ;l03lðYÞ; :l0klðYÞ . . . ;l0nlðYÞ�, where l0klðYÞ means team per-formance (Y) in rule k (k = 1, . . . ,n) and signal l (l = Red,Blue, Green). lSI(Y) represents the Safety Index (SI) ofteam performance (Y).

Step 6: Calculate the defuzzification. The most commonly usedmethod, the center of gravity (COG) method introducedby Mamdami in 1975, was applied in this system. Asshown in Fig. 10, the Safety Index (SI) of eight l0klðYÞ areSafety Index = Red and Safety Index = Blue where theinterval is between [yL = 1,yR = 8] and is divided into tareas. The crisp value is calculated as:

D¼1; q¼8�11þ1¼8

SI¼Xq

t¼1

YtlSIðYtÞ( ),Xq

t¼1

lSIðYtÞ

¼0:66�ð1þ2þ3Þþ0:33�4þ0�5þ0:32�ð6þ7Þþ0�80:66�3þ0:33þ0�2þ0:32�2

¼3:37

Step 7: Determine the signal of the Safety Index of the teamworkby Eqs. (11)–(13). The signal (Red, Bule, and Green) can bedecided by the maximum of the lRed(SI), lBlue(SI), andlGreen(SI). The results are shown in Table 3. From the sig-nal of the Safety Index, the proposed fuzzy inference sys-tem can diagnose the degree of safety in the currentteamwork in real-time. The purpose of this is to ensurethat the team members can maintain their teamperformance.

Fig. 9. Calculation of firing strength (ai).

1

1 2 3 4 5 6 7 8 9 10 Y

μ

Ly Ry

0.32

0.66

0

Blue Green Red

Fig. 10. Defuzzification of safety index.

432 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435

Table 3Safety Index of teamwork and signal type (modeling data)

Team No. RO ARO Supervisor Safety Index Signal

Real Estimative

1 0.34 0.54 0.34 3.37 2.41 Red2 0.65 0.24 0.24 2.21 2.14 Red3 0.22 0.22 0.22 2.11 2.2 Red4 0.71 0.26 0.26 2.10 2.1 Red5 0.69 0.69 0.69 8.42 7.89 Green6 0.83 0.83 0.83 8.42 8.41 Green7 0.35 0.04 0.04 2.11 2.18 Red8 0.89 0.89 0.89 8.42 7.77 Green9 0.58 0.58 0.58 6.44 5.22 Blue10 0.45 0.45 0.26 2.22 3 Red

Table 4Model 1: performance model validation (validation data)

Team No. x1 x2 x3 x4 x5 x6 x7 x8 Estimate

11 0.07 0.03 0.47 1.00 0.40 0.13 0.26 0.78 0.700.05 0.00 0.20 1.00 �0.05 0.21 0.23 0.20 0.700.11 0.01 0.05 0.00 0.11 �0.37 0.28 0.74 0.70

12 0.10 0.08 0.09 0.50 0.44 0.29 0.42 0.59 0.480.01 0.04 �0.39 0.00 �0.20 0.05 0.00 0.32 0.480.18 �0.03 0.08 �0.54 0.00 �0.53 0.67 �0.23 0.30

13 0.22 0.20 0.16 0.85 0.00 0.21 0.71 0.58 0.61�0.15 0.10 0.56 0.91 0.55 0.37 0.32 0.42 0.61

0.02 0.07 0.32 0.63 0.25 0.20 0.21 0.75 0.61

lRedðSIÞ ¼1; SI 6 2ð5� SIÞ=3; 2 6 SI 6 5

�ð11Þ

lBlueðSIÞ ¼ðSI� 2Þ=3; 2 6 SI 6 5ð8� SIÞ=3; 5 6 SI 6 8

�ð12Þ

lGreenðSIÞ ¼ðSI� 5Þ=3; 5 6 SI 6 81; SI P 8

�ð13Þ

where lRed(SI), lBlue(SI), and lGreen(SI) represent themembership degree of the Safety Index as Red, Blue, andGreen, respectively in the current team.

5.5. Model validation

Hollnagel (1996) argued that in the operator model, the datacollection is unrealistically simple, and so the results will be of lim-ited value. Thus, the models must be able to reflect the human sit-uation in an adequate way. Therefore, in this study, we furthervalidated the proposed (1) teamwork performance predictionmodel, and (2) teamwork real-time warning model (RTWM).

The nine participants’ data from the three teams were used asan input to validate the accuracy of the first proposed model, andthe results are shown in Table 4. The nine estimated values werevery close to the real values, with the real values all within the95% confidence intervals. Thus, this implies that the predictiveability of the first model can attain a 95% confidence level. In addi-tion, this result revealed that the data from the first model mayeffectively be used to construct the second model.

To validate the second model, the validation data from the threeteams was used as input for the fuzzy inference system. The resultof the validation is shown in Table 5. The estimated and real mem-bership values and signals that belong to Red, Blue, or Green cate-

d value Real value Low bound of 95% CI Upper bound of 95% CI

0.72 0.67 0.750.75 0.70 0.780.68 0.63 0.71

0.46 0.42 0.500.44 0.41 0.490.33 0.29 0.37

0.63 0.59 0.670.64 0.60 0.680.64 0.60 0.68

Table 5Model 2: Safety Index of teamwork and signal type (validation data)

Team No. RO ARO Supervisor Safety Index Membership Signal

Red Blue Green Estimated Real

11(e) 0.70 0.70 0.70 8.41 1 Green12(e) 0.48 0.48 0.30 2.72 1 Red13(e) 0.61 0.61 0.61 6.93 0.36 0.64 Green

11(r) 0.72 0.75 0.68 8.26 1 Green12(r) 0.46 0.44 0.33 3.24 0.59 0.41 Red13(r) 0.63 0.64 0.64 7.21 0.26 0.74 Green

(e) Represents the estimated value; (r) represents the real value.

Table 6Pearson correlation analysis between the real and estimated values of validation data

Correlations Real value Estimated value

Real value Pearson Correlation 1 0.999a

Significance (two-tailed) – 0.034N 10 10

Estimated value Pearson Correlation 0.999a 1Significance (two-tailed) 0.034 –N 10 10

a Correlation is significant at the 0.01 level (two-tailed).

S.-L. Hwang et al. / Safety Science 47 (2009) 425–435 433

gories were determined by Eqs. (11)–(13). The Pearson Correlationanalysis between the estimated and real Safety Index in Table 6were significant (p < 0.05) and the correlation coefficient turnedout to be very high (r = 0.99). Therefore, the second model canaccurately provide teams with the warning signal.

6. Discussion

6.1. Physiological indices on mental workload assessment

Physiological measurement and signals could be obtained inreal-time and recorded continuously in the present investigation(Luximon and Goonetilleke, 2001; Braarud, 2001; Laine and Bauer,2002; Wilson and Russell, 2003; Rubio and Diaz, 2004).

In this experiment, eight heart rate variety indices that wereaccomplished standards of the Task Force of the European Societyof Cardiology and the North American Society of Pacing and Elec-trophysiology in 1996 were collected and analyzed. Comparedwith previous studies that measure mental workload, the heartrate variability is more efficient than eye blink frequency, bloodpressure, electroencephalogram (EEG), and other tests. Althoughsubjective mental workload assessment in teamwork proved tobe insignificant, the physiological indices revealed that some heartrate variety indices among the team members were significant. Thefinding of heart rate variety indices among them indicated that thestandard deviation of the normal-to-normal intervals (SDNN), HRVtriangular index (HRVti), and total power (TP) between the assis-tant reactor operator and supervisor are insignificant; however,they are significant with the reactor operator. Furthermore, thestandard deviation of the normal-to-normal intervals (SDNN), thenumber of normal-to-normal interval pairs with over 50 m s differ-ence (pNN50), HRV triangular index (HRVti), the root mean squareof successive normal-to-normal interval differences (RMSSD), andtotal power (TP) of the heart rate variability indices of the assistantreactor operator and supervisor are higher than those of the reac-tor operator. These can be due to the following reasons: (1) eachteam member has a different responsibility for teamwork; and(2) the reactor operator has to operate more procedures than theassistant reactor operator and supervisor. These findings are con-sistent with previous heart rate variability studies that show as

task complexity increases, the heart rate variability decreases (TaskForce of the European Society, 1996; Hwang et al., 2008).

6.2. Team performance index assessment

Under the semi-automation task in this experiment, the errorrates and response time of teamwork were collected as perfor-mance indices. The team errors were classified into the followingcategories: (1) the reactor operator neglected to input the next tar-get value, (2) the assistant reactor operator and supervisor ignoredrecording parameters, (3) the recording parameters were differentbetween the assistant reactor operator and supervisor, and (4) allof the team members were not able to notice that the messagesof 29 break-points (BP1 � BP29) and the seven setting points ap-peared. The relationship between the error rates and response timeshowed that the longer the interval of the event arrival time, thehigher the error rates of the teams. This result was in agreementwith the curve of mental workload and performance as proposedby Veltman and Jansen (2006). This means that the lower the men-tal workload of team members, the lower the teamwork perfor-mance. Gauging from the responsibility of team members, thetasks of the reactor operator were more complex than those ofthe assistant reactor operator and supervisor. Thus, the mentalworkload of the assistant reactor operator and supervisor was low-er than normal. In addition, the special team of practical operatorsfrom the Fourth Nuclear Power Plant in Taiwan spends a lot of timechecking whether the parameters, instructions, and actions are cor-rect or not. Their accuracy rate is 100%, but they spend longer intheir response time.

6.3. Teamwork performance prediction model analysis

The results of the GMDH algorithm revealed that when the per-formance index is applied ðy1Þ

3=ffiffiffiffiffiy2p

, the R-square and correlationcoefficient are higher (refer Table 2). Although there were only30 data sets used in the model development, all predictive valuesfell in the 95% confidence intervals. The validation result of theproposed model is the same as that found in the work of Hwanget al. (2008).

6.4. Advantages, limitations, and application of the real-time warningmodel (RTWM)

Fuzzy inference system and fuzzy logic have been used for mod-eling in many fields (Becker et al., 1997; Ntuen, 1999; Moon et al.,2002; Liu and Su, 2006; Gregoriades and Sutcliffe, 2006; Rani et al.,2007; Yang et al., 2008). This study extended fuzzy applications todevelop a RTWM to detect the low performance of team members.The proposed RTWM could predict teamwork performance in real-time.

Furthermore, the RTWM have several advantages. One is thatthe manager can adjust the fuzzy numbers to set stricter or more

434 S.-L. Hwang et al. / Safety Science 47 (2009) 425–435

lenient management policy. Second, the model can be expandedinto a multiple input and multiple output system. Lastly, theGMDH algorithm can immediately combine individual perfor-mance values into team performance and then determine theteam’s safety level. However, some limitations exist in this model:(1) theoretical limitation, and (2) application limitation.

Firstly, the theoretical limitation with this work is that the cur-rently proposed real-time warning model (RTWM) did not focusedso much on team communication and coordination but consideredmore the monitoring of team cooperative tasks of low mentalworkload. Secondly, the model can effectively predict team’s per-formance and give the warning signal to team members based onthe simulator; however, in practice, there are still many task com-plexities to be considered (e.g., component complexity, coordina-tive complexity, dynamic complexity, and written manual oroperating procedures) (Wood, 1986; Park et al., 2003). Moreover,the application of the RTWM in practice should overcome thesetechnical limitations. For example, the data transferred from theCardio Tens portable device to the GMDH software and then tothe fuzzy inference system was a challenging task.

7. Conclusions

The control room of the Fourth Nuclear Power Plant (FNPP) ofTaiwan is a digitally controlled system. Automation is one of thecharacteristics of digital control systems. Many researches haveshown that automation may decrease operator’s workload, affordoperators to control more complex systems, and reduce the vari-ability of human performance (Young and Stanton, 1997; Parasur-aman, 2000; Metzger and Parasuraman, 2005; Liu and Su, 2006).This study simulated the teamwork of FNPP control room wherea reactor operator (RO), an assistant reactor operator (ARO), anda supervisor constitute a team to conduct the startup and monitor-ing tasks with the PCTran simulator. The experimental results re-vealed that (1) the operators’ psychological status changedaccording to the degree of complexity of the tasks, and (2) the teammembers made more errors when the interval of the event arrivaltime increased. Thus, in such computer-supported cooperativework (CSCW), one of the important things to avoid is the low men-tal workload of any team member which may result in human er-rors and accidents. Therefore, the teamwork performanceprediction model and the real-time warning model (RTWM) havebeen developed in this study. The proposed model can efficientlypredict teamwork performance in real-time to increase both sys-tem and human safety.

Acknowledgements

This research has been supported by the Institute of NationalScience Council of Taiwan (Project No. 962001INER005). Theauthors would like to thank the operators of Forth Nuclear PowerPlant engaged this study and the Institute of Nuclear Energy Re-search for providing such an opportunity.

References

Artman, H., 2000. Team situation assessment and information distribution.Ergonomics 43 (8), 1111–1128.

Baker, B.D., Richards, C.E., 1999. A comparison of conventional linear regression andneural network methods for forecasting educational spending. Economics ofEducation Review 18 (4), 405–415.

Becker, K., Thull, B., Käsmacher-Leidinger, H., Stemmer, J., Rau, G., Kalff, G.,Zimmermann, H.-J., 1997. Design and validation of an intelligent patientmonitoring and alarm system based on a fuzzy logic process model. ArtificialIntelligence in Medicine 11, 33–53.

Braarud, P., 2001. Subjective task complexity and subjective workload: criterionvalidity for complex team tasks. International Journal of Cognitive Ergonomics 5(3), 261–273.

Carvalho, P.V.R., Vidal, M.C.R., 2007. Nuclear power plant communications inNormative and actual practice. A field study of control room operator’scommunications. Human Factors and Ergonomics in Manufacturing 17 (1),43–78.

Dagdeviren, M., Yuksel, I., Kurt, M., 2008. A fuzzy analytic network process (ANP)model to identify faulty behavior risk (FBR) in work system. Safety Science 46(5), 771–783.

Desmond, P.A., Hoyes, T.W., 1996. Workload variation, intrinsic risk and utility in asimulated air traffic control task: evidence for compensatory effects. SafetyScience 22 (1–3), 87–101.

DiDomenico, A., Nussbaum, M.A., 2005. Interactive effects of mental and posturaldemands on subjective assessment of mental workload and postural stability.Safety Science 43, 485–495.

Greef, T., Dongen, K., Grootjen, M., Lindenberg, J., 2007. Augmenting cognition:reviewing the symbiotic relation between man and machine. In: The NinthInternational Conference on Human–Computer Interaction, Beijing, PR China,12–15 November, pp. 439–448.

Gregoriades, A., Sutcliffe, A.G., 2006. Automated assistance for human factorsanalysis in complex systems. Ergonomics 49 (12/13), 1265–1287.

Hale, A.R., Heming, B.H.J., Carthey, J., Kirwan, B., 1997. Modelling of safetymanagement systems. Safety Science 26 (1/2), 121–140.

Hale, A., Kirwan, B., Kjellen, U., 2007. Safe by design: where are we now? SafetyScience 45, 305–327.

Hollnagel, E., 1996. Reliability analysis and operator modeling. ReliabilityEngineering and System Safety 52, 327–337.

Hollnagel, E., 2003. Handbook of Cognitive Task Design. Erlbaum, Mahwah, NJ.Hwang, S.L., Yau, Y.J., Lin, Y.T., Chen, J.H., Huang, T.H., Yenn, T.C., Hsu, C.C., 2008.

Predicting work performance in nuclear power plants. Safety science 46 (7),1115–1124.

Ivakhnenko, A.G., 1968. Group method of data handling – rival of method ofstochastic approximation. Soviet Automatic Control 13 (1), 43–55.

Ivakhnenko, G.A., 1995. Self-organization of neuronet with active neurons foreffects of nuclear test explosions forecasting. System Analysis ModelingSimulation (SAMS) 20, 107–116.

Ivakhnenko, A.G., Ivakhnenko, G.A., Müller, J.A., 1993. Self-organization of optimumphysical clustering of the data sample for weakened description and forecastingof fuzzy objects. Pattern Recognition and Image Analysis 3 (4), 415–421.

Kageyama, T., Kabuto, M., 1995. Combined effects of mental workload and followingexposure to road traffic noise on sympathetic activities assessed by a spectralcomponent of heart rate variability. Arch Complex Environ Studies 7, 13–20.

Kamada, T., Sato, N., Miyake, S., Kumashiro, M., Monou, H., 1992a. Power spectralanalysis of heart rate variability in type as during solo and competitive mentalarithmetic task. Journal of Psychosomatic Research 36 (6), 543–551.

Kamada, T., Miyake, S., Kumashiro, M., 1992b. Power spectral analysis of heart ratevariability in Type As and Type Bs during mental workload. PsychosomaticMedicine 54 (4), 462–470.

Kim, S., Kim, J., Lee, C.B., 2001. Fuzzy decision support system to the prediction ofozone concentrations. In: IEEE Conference, Pusan, Korea.

Klir, G.J., Yuan, B., 1995. Fuzzy Sets and Fuzzy Logic: Theory and Applications.Prentice Hall.

Kruse, R., Gebhardt, R., Klawonn, F., 1994. Foundations of Fuzzy Systems. Wiley,Chichester.

Kulkarni, A.D., 2001. Computer Vision and Fuzzy-Neural Systems. Prentice Hall.Laine, T.I., Bauer, K.W., 2002. Selection of input features across subjects for

classifying crewmember workload using artificial neural networks. IEEETransactions on Systems 32 (6), 691–704.

Liu, C.L., Su, K.W., 2006. A fuzzy logical vigilance alarm system for improvingsituation awareness and trust in supervisory control. Human Factors andErgonomics in Manufacturing 16 (4), 409–426.

Luximon, A., Goonetilleke, R.S., 2001. Simplified subjective workload assessmenttechnique. Ergonomics 44 (3), 229–243.

Madala, H.R., Ivakhnenko, A.G., 1994. Inductive Learning Algorithms for ComplexSystems Modeling. CRC Press Inc, Boca Raton.

Mamdami, E.H., Assilina, S., 1975. An experiment in linguistic synthesis with a fuzzylogic controller. International Journal of Man–Machine Studies 7 (1), 1–13.

Marseguerra, M., Zio, E., 2006. Quantitative developments in the cognitive reliabilityand error analysis method (CREAM) for the assessment of human performance.Annals of Nuclear Energy 33, 894–910.

Mclntyre, R.M., Strobel, K., Hanner, H., Cunningham, A., Tedrow, L., 2003. Toward anUnderstanding of Team Performance and Team Cohesion over Time through theLens of Time Series Analysis. Army Research Institute (ARI) Research Note2003–2007.

McNames, J., Thong, T., Goldstein, B., 2003. Reliability and accuracy of heart ratevariability metrics versus ECG segment duration. Engineering in Medicine andBiology Society. Proceedings of the 25th Annual International Conference of theIEEE 1 (17–21), 212–215.

Metzger, U., Parasuraman, R., 2005. Automation in future air traffic management:effects of decision aid reliability on controller performance and mentalworkload. Human Factors 47 (1), 35–49.

Mikulka, P.J., Scerbo, M.W., Freeman, F.G., 2002. Effects of a biocybernetic system onvigilance performance. Human Factors 44, 654–664.

Moon, B.S., Lee, H.C., Lee, Y.H., Park, J.C., Oh, I.S., Lee, J.W., 2002. Fuzzy systems toprocess ECG and EEG signals for quantification of the mental workload.Information Sciences 142, 23–35.

Moray, N., 1988. Mental workload since 1979. International Review of Ergonomics2, 123–150.

S.-L. Hwang et al. / Safety Science 47 (2009) 425–435 435

Nishikido, T.K., Kurokawa, Y.H., Kobayashi, H.I., Kabuto, T.K., 1997. Effects of obesity,current smoking status, and alcohol consumption on heart rate variability inmale white-collar workers. International Archives Occupation and EnvironmentHealth 69, 447–454.

Ntuen, C.A., 1999. The application of fuzzy set theory to cognitive workloadevaluation of electronic circuit board inspectors. Human Factors andErgonomics in Manufacturing 9 (3), 291–301.

Parasuraman, R., 2000. Designing automation for human use: empirical studies andquantitative models. Ergonomics 43 (7), 931–951.

Park, J., Jung, W., Kim, J., Ha, J., 2003. Step complexity measure for emergencyoperating procedures-determining weighting factors. Nuclear Plant Operationsand Control 143, 290–308.

Pavel, N., Miroslav, S., 2003. Modeling of student’s quality by means of GMDHalgorithms. Systems Analysis Modeling Simulation 43 (10), 1415–1426.

Rani, P., Sarkar, N., Adams, J., 2007. Anxiety-based affective communication forimplicit human–machine interaction. Advanced Engineering Informatics 21,323–334.

Rentsch, J.R., Klimoski, R.J., 2001. Why do ‘great minds’ think alike?: antecedents ofteam member schema agreement. Journal of Organization Behavior 22, 107–120.

Rubio, S., Diaz, E., 2004. Evaluation of subjective mental workload: a comparison ofSWAT, NASA-TLX, and workload profile methods. Applied Psychology: Aninternational Review 53 (1), 61.

Sammer, G., 1998. Heart period variability and respiratory changes associated withphysical and mental load: non-linear analysis. Ergonomics 41, 746–755.

Sarycheva, L., 2003. Using GMDH In ecological and socio-economical monitoringproblems. Systems Analysis Modeling Simulation (SAMS) 43 (10), 1409–1414.

Scerbo, M.W., Freeman, F.G., Mikulka, P.J., 2001. The Efficacy of PsychophysioligicalMeasures for Implementing Adaptive Technology. NASA/TP-2001-211018.

Schupp, B., Hale, A., Pasman, H., Lemkovitz, S., Goossens, L., 2006. Design support forthe systematic integration of risk reduction into early chemical process design.Safety Science 44, 37–54.

Sebok, A., 2000. Team performance in process control: influences of interface designand staffing levels. Ergonomics 43 (8), 1210–1236.

Shu, Y., Furuta, K., 2005. An inference method of team situation awareness based onmutual awareness. International Journal Cognition, Technology, and Work 7 (4),272–287.

Skitka, L.J., Mosier, K.L., Burdick, M., Rosenblatt, B., 2000. Automation bias anderrors: are crews better than individuals? The International Journal of AviationPsychology 10 (1), 85–97.

Stanton, N., Hedge, A., Brookhuis, K., Salas, E., Hendrick, H., 2005.Handbook of Human Factors and Ergonomics Methods. CRC Press,Boca Raton, FL.

Task Force of The European Society of Cardiology and The North American Society ofPacing and Electrophysiology, 1996. Heart rate variability: standards ofmeasurement, physiological interpretation, and clinical use. European HeartJournal 17, 354–381.

Tattersal, A.J., Hockey, G.R.J., 1995. Level of operator control and changes in heartrate variability during simulated flight maintenance. Human Factor 37, 682–698.

Veltman, J.A., Jansen C., 2006. The Role of Operator State Assessment in Adaptive.TNO Report, TNO-DV3 2005 A245.

Wang, L.X., Mendel, J.M., 1992. Generating fuzzy rules by learning from examples.In: Proceedings of the IEEE International Symposium on Intelligent Control, vol.22(6). Arlington, pp. 1414–1427.

Wilson, G.F., Russell, C.A., 2003. Real-time assessment of mental workload usingpsychophysiological measures and artificial neural networks. Human Factors 45(4), 635–643.

Wood, R.E., 1986. Task complexity: definition of the construct. OrganizationalBehavior and Human Decision Processes 37, 60–82.

Ward Systems Group (WSG), 1995. Neuroshell 2. User’s Guide. WSG, Frederick, MD.Xie, B., Salvendy, G., 2000. Review and reappraisal of modeling and predicting

mental workload in single- and multi-task environments. Work and Stress 14(1), 74–99.

Yang, G., Lin, Y., Bhattacharya, P., 2008. Multimodality inferring of human cognitivestates based on integration of neuron-fuzzy network and information fusiontechniques. EURASIP Journal on Advances in Signal Processing, Article ID371621, 14 p., doi:10.1155/2008/371621.

Young, M.S., Stanton, N.A., 1997. Automotive automation: investigating the impacton drivers’ mental workload. International Journal of Cognitive Ergonomics 1(4), 325–336.

Zadeh, L., 1965. Fuzzy sets. Information and Control 8 (3), 338–353.