Wavelet kernel entropy component analysis with application to industrial process monitoring

Wavelet kernel entropy component analysis with applicationto industrial process monitoring

Yinghua Yang n, Xiaole Li, Xiaozhi Liu, Xiaobo ChenCollege of Information Science and Engineering, Northeastern University, Shenyang, Liaoning 110819, China

a r t i c l e i n f o

Article history:Received 25 February 2014Received in revised form10 May 2014Accepted 16 June 2014Communicated by W. Yu

Keywords:Kernel entropy component analysisWavelet transformProcess monitoringFault identificationTE process

a b s t r a c t

Aiming at the features that modem industrial processes always have some characteristics of complexityand nonlinearity and the process data usually contain both Gaussion and non-Gaussion information atthe same time, a new process performance monitoring and fault detection method based on wavelettransform and kernel entropy component analysis (WT-KECA) is proposed in this paper. Unlike otherkernel feature extraction methods, this method chooses the best principal component vectors accordingto the maximal Renyi entropy rather than judging by the top eigenvalues and eigenvectors of the kernelmatrix simply. Besides, it can denoise and anti-disturb due to the application of wavelet transform. Theproposed method is applied to process monitoring in the Tennessee Eastman (TE) process and the faultidentification is realized. The simulation results indicate that the proposed method is more feasible andefficient in comparing to KPCA method.

& 2014 Elsevier B.V. All rights reserved.

1. Introduction

The monitoring of industrial processes and the detection offaults in those processes are especially significant procedures asthe industrial process control system tends to be large-scale andcomplicated. The actual production data, however, inevitably hasproperties of noises, random disturbances. There exist some datatransformation methods to tackle these problems. Among them,Principal Component Analysis (PCA) is a powerful technique forextracting structure from possibly high-dimensional data sets andwidely utilized for process monitoring and fault diagnosis onaccount of its ability to handle high-dimensional, noisy, and highlycorrelated data. MacGregor and Kourti [1] established a PCA modelfrom the training data and detected the abnormal behavior ofonline processes. However, PCA is a linear transformation methodwhose performance would degrade the monitoring performancegreatly for some complicated cases in chemical industry processwith particularly nonlinear characteristics [2].

To solve the problem posed by nonlinear data, Scholkopfproposed kernel principal component analysis (KPCA) method[3]. The main idea of KPCA is to map the input space into a featurespace via nonlinear mapping and then perform PCA in a higherdimensional feature space. It has proven to be a very effective faultdiagnostic method in process monitoring because of the mainadvantage that it does not involve nonlinear optimization [4].

However, due to the complexity of industry, the application ofKPCA in process monitoring is not very good for some complicatedindustrial processes fault and may cause false alarms.

Kernel entropy component analysis (KECA) is a new methodof data transformation and dimensionality reduction which hasbeen proposed by Robert Jenssen recently [5]. KECA is founded oninformation theory and tries to preserve the maximum Renyiquadratic entropy estimated via Parzen window rather thandepend on the second order statistics of the data set [6]. As aresult, no limitation of Gaussian-like assumption is involved beforewe apply the KECA method. KECA may produce strikingly differenttransformed data sets whose data transformation is achieved byprojecting onto the kernel PCA axes that contribute to the max-imum entropy estimate of the input space data set in comparing toKPCA method with data transformation corresponding to the topeigenvalues of the kernel matrix. Some scholars have applied KECAon face recognition, audio emotion recognition, data clusteringand denoising techniquewhich lead to better results than PCA andKPCA [7–9]. However, there were few reports about the reseach onthe KECA applied in process monitoring. In addition, the actualproduction data of chemical process inevitably contain randomand gross errors due to sensor noise, disturbances, instrumentdegradation, and human errors. So when we only apply KECA forprocess monitoring and fault detection, it will impact the effectof process information treatment or analysis and reduce theconfidence degree of outcome by using these contaminated data.Hence straightly using KECA is not fit for fault detection very well.

In order to improve the effect of KECA method application inthe field of fault identification, wavelet transform (WT) have

Contents lists available at ScienceDirect

journal homepage: www.elsevier.com/locate/neucom

Neurocomputing

http://dx.doi.org/10.1016/j.neucom.2014.06.0450925-2312/& 2014 Elsevier B.V. All rights reserved.

n Corresponding author.E-mail address: [email protected] (Y. Yang).

Please cite this article as: Y. Yang, et al., Wavelet kernel entropy component analysis with application to industrial process monitoring,Neurocomputing (2014), http://dx.doi.org/10.1016/j.neucom.2014.06.045i

Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

www.sciencedirect.com/science/journal/09252312

www.elsevier.com/locate/neucom

http://dx.doi.org/10.1016/j.neucom.2014.06.045



mailto:[email protected]





shown that wavelet is an efficient tool for noise removal of noisysignal and it has found applications in a variety of fields inbiomedical signal processing. Wavelet transform is a kind oftime–frequency analysis and it provides a useful alternative toFourier methods in the enhancement of nonlinear data. Wavelethas characteristics of well time frequency localization, specialdenoising ability and convenient extracting weak signals for signalprocessing [10] so it has found applications in a variety of fields insignal analysis, image processing, data compression, and processmodeling.

In this paper, we apply KECA method to dynamic nonlinearprocess monitoring and it shows a better monitoring performancethan other approaches in the simulation of TE process. What'smore, according to the above discussion, this paper modifies KECAwith wavelet (WT-KECA), and proposes a novel process monitor-ing algorithm. This method takes both advantaged of KECA and thewavelet method and it also shows a better performance.

The remaining of this paper is organized as follows: Section 2explains the KECA and wavelet analysis algorithms. In Section 3,further discussion of WT-KECA on Process monitoring is describedand Section 4 is the simulation process of TE and discussion.Finally, Section 5 concludes the paper.

2. Wavelet kernel entropy component analysis

According to the discussion in the previous section, theimproved method based on wavelet analysis and kernel entropycomponent analysis is proposed in this paper, in which the originalsignal is firstly decomposed by wavelet analysis, then the kernelentropy component analysis method is applied to the prepro-cessed data. In the following section, the basic principle of theKECA and wavelet analysis will be introduced respectively.

2.1. Kernel entropy component analysis

The Renyi quadratic entropy [11] is given by

HðpÞ ¼ � logZ

p2ðxÞdx ð1Þ

where pðxÞ is the probability density function of the data set, orsample, D¼ x1; :::; xN . Because the logarithm is a monotonic func-tion, we can consider the following quantity

VðpÞ ¼Z

p2ðxÞdx ð2Þ

In order to estimate VðpÞ, and hence HðpÞ, we may invoke aParzen window density estimator described as

p̂ðxÞ ¼ 1N

∑N

i ¼ 1Kσðx; xiÞ ð3Þ

here, Kσðx; xiÞ is the so-called Parzen window, or kernel, centeredat xi and with a width governed by the parameter σ.

Using the sample mean approximation of the expectation operator,we have

V̂ðpÞ ¼ 1

N2 ∑N

i ¼ 1∑N

jKσðxi; xjÞ ¼

1

N21TK1 ð4Þ

Here, the element ði; jÞ of the N�N kernel matrix K is Kσðxi; xjÞand 1 is an (N � 1) vector containing all ones.

The Renyi entropy estimator may be expressed in terms of theeigenvalues and eigenvectors of the kernel matrix, which may bedecomposed as K¼ EDλET , where Dλ is a diagonal matrix storingthe eigenvalues λ1; :::; λN and E is a matrix with the corresponding

eigenvectors e1; :::; eN as columns. Rewriting (4), we then have

V̂ðpÞ ¼ 1

N2 ∑N

i ¼ 1ð

ffiffiffiffiλi

peiT1Þ2 ð5Þ

Each term in Eq. (5) will contribute to the entropy estimate.This means that certain eigenvalues and eigenvectors will con-tribute more to the entropy estimate than others since the termsdepend on different eigenvalues and eigenvectors. The eigenvaluesand eigenvectors selected are the first l largest contribution to theentropy estimate in KECA so that the cumulative contribution rateof the selected Renyi entropy reaches 85% of all the Renyi entropy.

In KPCA, the nonlinear map from input space to feature space isgiven by ϕ : Rd-F such that xt-ϕðxtÞ; t ¼ 1;…;N.Let Φ¼ ½ϕðx1Þ;…;

ϕðxNÞ� and ui is the feature space principal axe. The projection of Φonto theith principal axis ui in the kernel feature space is definedas Puiϕ¼ ffiffiffiffi

λip

eTi .Eq. (5) therefore reveals that the Renyi entropyestimator is composed of projections onto all the kernel PCA axes.Certainly, only a principal axis ei for which λia0 and eTi 1a0contributes to the entropy estimate. Hence, ei is composed of asubset of KPCA axes but not necessarily those corresponding to thetop l eigenvalues.

The kernel entropy component analysis procedure, as describedabove, is summarized as follows:

(1) Select the kernel function and get the kernel matrix K;(2) Compute the eigen-decomposition of K : K¼ EDλET ;(3) Select the first l largest contribution to the entropy estimate

in KECA;(4) Calculate the kernel feature space data points, Φeca ¼D1=2

l ETl

and the components TN�l ¼KN�NEN�l.

2.2. Wavelet transform for denoising

Wavelet analysis is a new method of time–frequency analysis,it has the characteristic of multi-resolution analysis and focus onthe details of signal points to make the time–frequency domainanalysis thus it has been called the “mathematical microscope”. Inthis section, we will introduce the basic theory of wavelet.

2.2.1. Discrete wavelet transformIn discrete wavelet analysis, xðtÞ is decomposed on different

scale s as follows:

xðtÞ ¼ ∑K

j ¼ 1∑1

k ¼ �1djðkÞψ j;kðtÞþ ∑

1

k ¼ �1aK ðkÞϕK;kðtÞ ð6Þ

where ψ j;kðtÞ are discrete analysis wavelets and ϕK;kðtÞ are discretescaling functions, djðkÞ are the detailed signals (wavelet coeffi-cients) at scale 2j and aK ðkÞ is the approximated signal (scalingcoefficients) at scale 2K . The idea of discrete wavelet analysis ispresented by means of a wavelet decomposition tree.

The discrete wavelet transform can be implemented by thescaling and wavelet filters

hðnÞ ¼ 1ffiffiffi2

p ϕðtÞ;ϕð2t�nÞ� �gðnÞ ¼ 1ffiffiffi

2p ψðtÞ;ψ ð2t�nÞ� �¼ ð�1Þnhð1�nÞ ð7Þ

The estimation of the detail signal at level j will be done byconvolving the approximate signal at level j�1 with the coeffi-cients gðnÞ. Convolving the approximate signal at level j�1 withthe coefficients hðnÞ gives an estimate for the approximate signalat level j. The decomposition scheme involves retaining everyother sample of the filter output.

Y. Yang et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎2





Let us define the scaling function W0ðtÞ ¼ ϕðtÞ and the waveletfunction W1ðtÞ ¼ ψðtÞ. Then we can write functions WmðtÞ;m¼ 0;1;2;⋯; as

W2mðtÞ ¼ 2 ∑2N�1

n ¼ 0hðnÞWmð2t�nÞ

W2mþ1ðtÞ ¼ 2 ∑2N�1

n ¼ 0gðnÞWmð2t�nÞ ð8Þ

WmðtÞ includes the function set which is made up of the scalingfunctions and the wavelet functions. Form wavelet packet, thefrequency band of signal can be divided into multiple levels, andwavelet packet improves the time–frequency resolution compar-ing to wavelet transform. Therefore, wavelet packet has a moreextensive application value. This method can further deintegratethe part which is not divided by wavelet. The three layer waveletpacket decomposition of one-dimensional signal as shownin Fig. 1.

2.2.2. Denoising with wavelet thresholdingUsing wavelet denoising is an important application of wavelet

analysis in engineering. There are four main wavelet denoisingmethods [12]: the wavelet decomposition and reconstructionmethod, the nonlinear wavelet threshold denoising method, thetranslation invariant denoising method and the wavelet transformmodulus maxima method. Owing to nonlinear wavelet transformthreshold value method having merits of nearly complete restrain-ing noises, comprehensive suitability and fast calculating speed[13], this method is used to this paper.

In the engineering, additive noise model of the one-dimensionalsignal can be expressed as the following forms:

sðkÞ ¼ f ðkÞþεeðkÞ; k¼ 0;1;…;n�1 ð9Þ

where sðkÞ is the noisy signal, f ðkÞ is a true signal, eðkÞ �Nð0;1Þ isthe distribution of the Gaussian white noise, ε is the deviation forthe noise signal.

The specific steps of the nonlinear wavelet transform thresholdvalue method are divided into three steps. First, noisy signal aredecomposed by wavelet and decomposed level N is determined, sothe wavelet coefficients are obtained. Then the high coefficient ofwavelet decomposition is disposed by quantifying threshold value.Finally, wavelet reconfiguration is obtained by No. N level lowfrequency of wavelet decomposition and quantifying 1-N levelhigh frequency coefficients.

The key step of denoising process is how to choose thresholdvalue and treat threshold value. There are two main thresholdselection methods: soft threshold and hard threshold. The softthreshold is adopted in this paper and expressed as the followingforms:

y¼signðxÞðjxj�TÞ; jxj4T0; else

(ð10Þ

3. Wavelet kernel entropy component analysis (WT-KECA) forprocess monitoring

We only discuss the basic ideas of KECA and wavelet analysis inSection 2. Thereafter, the application on process monitoring ofWT-KECA needs further discussion.

3.1. Choose kernel function

The kernel function choice is the key problem of kernelmethod. There are two kinds of kernel functions: positive semi-definite and indefinite kernel. The properties of the kernel spacedepend on the choice of kernel function[14]. Previously, almostall research on kernel methods in machine learning focuses onfunctions Kσðx; xiÞ which are positive semi-definite. That is, itfocuses on kernels which satisfy Mercer's condition and whichconsequently can be seen as scalar products in some Hilbert space.The positive semi-definite (psd) functions such as Mercer kernelgain the successful in recent years with the development ofsupport vector machine (SVM). More recently, however, someresearchers have pointed out that the positivity property of thekernel is quite restrictive [15]. Then some first try to develop atheory for using indefinite kernel functions instead of Mercerkernels. Spectral clustering [16] showed that the normalized cutmethod may work well also for some indefinite kernel functions.

Since the theoretically optimal Parzen window is in factindefinite. Therefore, we adopt an indefinite kernel function—Epanechnikov kernel [17] in KECA:

Wσðx; xtÞ ¼34

1�jjx�xt jj2σ2

� �ð11Þ

Through the Epanechnikov kernel, the Renyi entropy has beenbetter estimated.

3.2. Determine principal component numbers (PCs)

In the process of information extraction, reasonable determin-ing the PCs is very important. Certainly, the more PCs chosen, themore accurate the model will be. However, this increases thecomplexity of the analysis and diagnosis and it cannot removethe noise effectively. Correspondingly, the PCs selected too littlecannot be sufficient to extract the information of the original dataspace and it increases the error rate of the analysis and diagnosis.Therefore, in the process of information extraction, we mustdetermine the PCs reasonably.

Generally, there are two methods to determine the PCs: theaverage method and the empirical method. The average method isto calculate the mean of characteristic root λ and then choose theprincipal components corresponding to the characteristic rootswhich are greater than λ. The empirical method is based on thecumulative contribution rate to determine the number of principalcomponents.

In actual engineering projects, we usually adopt empiricalmethod and so this method has been applied in this paper. Thechoice of the PCs is according to the cumulative contribution rateof Renyi entropy. Through the simulation experiment, we can getthe cumulative contribution rate of 85% when the number ofprincipal component is twenty-five. However, it needs at leastthirty-five principal components to achieve the same results inKPCA and this also shows the superiority of KECA.

3.3. Online data preprocessing by wavelet

In practical application, the greatest disadvantage of waveletdenoising is that it cannot be utilized online mainly due to thetwo reasons. First, wavelet translation is noncausal in nature and

S

CA1

CD1

CD2

CA2

CA3

CD3

Fig. 1. Tree of wavelet decomposition.

Y. Yang et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎ 3





requires future measured data for calculating the current waveletcoefficient. Second, the dyadic discretization of the wavelet para-meters requires a signal of dyadic length for the wavelet decom-position which also leads to a time delay. Consequently, On-LineMultiscale Rectification (OLMS) was proposed by Nounou andBakshi [18]. OLMS is based on multiscale rectification of data ina moving window of dyadic length, as shown in Fig. 2. In Fig. 2, thenumbers in each row are the sampling data points and the lastpoint corresponds to the new sampling points which is about to bewavelet reconstructed. When new measured data are available,we should move the window in time to include the most recentmeasurement while maintaining the maximum dyadic windowlength. The measurements in each window are rectified by thenonlinear wavelet transform threshold value method.

OLMS method retains the benefits of the wavelet decomposi-tion in each moving window, while allowing each measurement tobe rectified on-line. The OLMS methodology can be summarized asfollows:

(1) Decompose the measured data within a window of dyadiclength using a causal boundary corrected wavelet filter andthreshold the wavelet coefficients.

(2) Reconstruct the rectified signal and retain only the last datapoint of the reconstructed signal for on-line use.

(3) When new measured data are available, move the window intime to include the most recent measurement while main-taining the maximum dyadic window length.

3.4. Calculate statistic in process monitoring

WT-KECA monitoring method is similar to that using KPCA inthe Hotelling's T2 statistic and the SPE statistic. Therefore, The T2

and SPE statistics are used as the method for data monitoring. T2 isthe sum of the normalized squared scores, and is defined as

T2 ¼ ½t1;…; tp�Λ�1½t1;…; tp�T ð12Þwhere ½t1;…; tp� is the squared scores, and Λ�1 is the diagonalmatrix of the inverse of the eigenvalues associated the retainedPCs. The control limit can calculated using the F distribution

T2p;N;a ¼

pðN�1ÞN�p

Fp;N;a ð13Þ

The SPE statistic is

SPE¼ jjϕðxÞ� ϕ̂pðxÞjj2 ¼ ∑l

j ¼ 1t2j � ∑

p

j ¼ 1t2j ð14Þ

where pis the number of PCs, and lis the number of samples. ϕ̂pðxÞis a reconstructed feature vector where ϕ̂pðxÞ ¼ Σp

k ¼ 1tkvk,vkis theeigenvector.

3.5. Fault identification strategies

Once a fault occurs, it is important that operators analyze thereason of the failure and identify the corresponding faulty variablequickly. There exists certain linear relationship between faultyvariables and monitoring variables in PCA and we can calculate thecontribution of faulty variables in fault-contribution plot easily.However, in KPCA or KECA, we cannot obtain the clear mappingrelationship in comparing with linear method. Through furtherstudying, however, some scholars still have discovered that thereis a certain correlation between faulty variable and nonlinearprincipal component [19]. Therefore, we have the calculationmethod of contribution amount:

CONTj ¼ ∑p

i ¼ 1jconti;jj ð15Þ

where conti;j ¼ tiTxj=Ri and p is the number of principal compo-nents in the feature space. ti is the ith nonlinear principalcomponent and xj is the jth variable in faulty variables. Ri is thecorresponding Renyi entropy.

So, we can identify the fault variable according to the variablewhich has the most contribution in contribution rate chart.

3.6. Process monitoring based on WT-KECA

In practical applications, we adopt both the KECA and WT-KECAto process monitoring. The monitoring procedure has two pro-cesses: off-line training and on-line monitoring. In off-line training'sprocedure, the PCs, T2 and SPE are calculated by normal operatingdata and in on-line monitoring, monitoring statistics are calculatedby the testing data to judge whether a fault happens. The proce-dures of process monitoring based onWT-KECA are given as below:

A. Off-line training

(1) Acquire the training data and denoising them before normalizingthem.

(2) Calculate the kernel matrix by Eq. (11) and then obtain theeigenvalues and corresponding eigenvectors to calculateentropy estimate.

(3) Select PCs according to the largest l contribution to the entropyestimate.

(4) Determine the control limits of the T2 and SPE according toEqs. (13) and (14).

B. on-line monitoring

(1) Collect the real-time monitoring data and denoising thembefore normalizing them.

(2) Calculate the kernel matrix by Eq. (11) and then obtain theeigenvalues and corresponding eigenvectors to calculate entropyestimate.

(3) Select PCs according to the largest l contribution to the entropyestimate.

(4) Calculate the monitoring statistics (T2 and SPE) of the test dateaccording to Eqs. (12) and (14).

(5) Monitor whether T2 or SPE exceeds its control limit calculatedin the modeling procedure. If it exceeds the control limit, afault might happen.

The detailed flowchart of the WT-KECA in fault identificationis shown in Fig. 3. Subfigure (a) is off-line modeling process andsubfigure (b) is on-line monitoring process.

1 2

3

1

2

3

7

1

2

2 3 4

3 4 5

4 5 6

4 5 6

2 3 4 5 6 7 8

Fig. 2. Scheme of the OMLS gliding windows.






4. Experiments and discussion

4.1. Process description

The TE process simulator was created by the Eastman ChemicalCompany to provide a realistic industrial process for evaluatingprocess control and monitoring methods. The process consists of

five major unit operations: a reactor, a condenser, a recyclecompressor, a separator, and a stripper and there are 21 processfaults in TE process. Fig. 4 shows the diagram of TE process and thespecified fault is listed in Table 1.

The process has 41 measured variables (22 continuous processmeasurements and 19 composition measurements) and 12 mani-pulated variables. Measurements are taken every 3 min. In our

Normalize the denoised data

Compute the kernel matrix anddetermine the eigenvalues and

eigenvectors

Compute Load vector and makethem in descending order

According to the Renyi entropy

Determine the PCs andgenerate the score matrix

Normalize the eigenvector anddetermine the PCs

Determine the control limits ofthe T2 and SPE

N

Y

Constructe the KECA model

Calculating the nonlinear PCs

Calculate the monitoring statistics (T2and SPE)

Monitor whether T2 or SPEexceeds its control

Complete the faultrecognition

Acquire training data

Remove the noise by waveletthresholding

Collect the real - time monitoring data

Remove the noise by waveletthresholding

Normalize the denoised data

Fig. 3. Flow chart of fault diagnosis with WT-KECA.

Fig. 4. A diagram of the Tennessee Eastman process.






experiments, there are 52 process variables (the agitation speed ofthe reactor is excluded). In the simulation process, 500 observa-tions under normal working conditions are selected as the trainingdata; 960 observations are selected to be the testing data undereach fault condition and the introduction of the fault start atsample 161. The training and test data set we used in the study canbe downloaded from http://brahms.scs.uiuc.edu.

4.2. Experiment results and discussion

The KPCA method, KECA method and WT-KECA method areapplied to TE process respectively to compare and verify their

performances. In the simulation, the choice of kernel types andkernel parameters are the same in these three methods. That is,their kernel type is Epanechnikov kernel and their kernel para-meter is σ2 ¼ 7000. Considering that the cumulative contributionrate should be higher than 85%, the selected PCs is 35 in KPCAmethod but the KECA and WT-KECA methods only need 25.Additionally, in WT-KECA method, the length of the initial movingwindow is 128. The use of wavelet is db5, level 5 and we adoptthe soft threshold mentioned above. All the required parameterssetting is according to the experiment and when the parameterstake these values, the monitoring of fault shows better perfor-mance and highly reliability.

Table 1Process fault for the Tennessee Eastman process simulator.

Fault Description Type

1 A/C feed radio, B composition constant (Stream 4) Step2 B composition, A/C radio constant (Stream 4) Step3 D feed temperature (Stream 2) Step4 Reactor cooling water inlet temperature Step5 Condenser cooling water inlet temperature Step6 A feed loss (Stream 1) Step7 C header pressure loss—reduced availability (Stream 4) Step8 A,B,C feed composition (Stream 4) Random variation9 D feed temperature (Stream 2) Random variation

10 C feed temperature (Stream 4) Random variation11 Reactor cooling water inlet temperature Random variation12 Condenser cooling water inlet temperature Random variation13 Reaction kinetics Slow drift14 Reactor cooling water valve Sticking15 Condenser cooling water valve Sticking16 Unknown Unknown17 Unknown Unknown18 Unknown Unknown19 Unknown Unknown20 Unknown Unknown21 The valve for Stream 4 was fixed at the steady state position Constant position

0 200 400 600 800 10000

0.2

0.4

0.6

0.8

KE

CA

-SP

E

sample

0 200 400 600 800 10000

100

200

300

KE

CA

-T2

sample

0 10 20 30 40 50 600

20

40

60

KE

CA

-con

tribu

tion

rate

(%)

variable

0 200 400 600 800 10000

10

20

30

KP

CA

-SP

E

sample

0 200 400 600 800 10000

100

200

300

KP

CA

-T2

sample

0 10 20 30 40 50 600

10

20

30

KP

CA

-con

tribu

tion

rate

(%)

variable

0 200 400 600 800 10000

1

2

3

WT-

KE

CA

-SP

Esample

0 200 400 600 800 10000

1000

2000

3000

4000

WT-

KE

CA

-T2

sample

0 10 20 30 40 50 600

20

40

60

80

WT-

KE

CA

con

tribu

tion

rate

(%)

variable

Fig. 5. Monitoring charts of fault 4 for KPCA, KECA and WT-KECA.



http://brahms.scs.uiuc.edu




In the monitoring charts, the left column is for KPCA approachand the subfigure in each row corresponds to the SPE, T2 andvariable contribution rate charts from top to bottom; the middlecolumn is for the KECA approach and the far-right column is forthe WT-KECA approach. The respective subfigures are the corre-sponding statistical figures. Figs. 5 and 6 demonstrate the mon-itoring performances for faults 4 and 5 of KPCA, KECA and theproposed WT-KECA respectively.

For fault 4, the monitoring results of KPCA, KECA and WT-KECAare shown in Fig. 5. The cause of fault 4, which is correlated withvariable 51, is that a step change involving of cooling water inlettemperature in the reactor. From the first row in Fig. 5, we cansee the SPE statistics of the three methods are overtop respectivecontrol limits from sample 161. Hence, as for SPE, WT-KECA hasthe same high fault detection rate as KPCA and KECA. However,from the second row in Fig. 5, the T2 statistics of KPCA have poorperformance but the T2 statistics of KECA are mostly beyond thecontrol respective limits from sample 161. What's more, the T2

statistics of WT-KECA change more obvious than KPCA and KECAwhen the fault 4 occurs. Thus, this shows the superiority of KECAand WT-KECA preliminarily. Meanwhile, in the third row, it can beseen that the accurate fault variable (variable 51) is determined bythe three methods but the contribution rate fault 4 in KECA is54.45% and increased by 30% than the KPCA method. More thanthat, the Fault contribution chart of WT-KECA is more clear thanKECA and KPCA, and the contribution rate reaches 74%. In a word,the KECA method demonstrates better monitoring performancethan the KPCA method form fault 4.

In the case of fault 5, it is due to the step disturbance incondenser cooling water inlet temperature. In this mode, the flowrate of the outlet stream from the condenser increases, which isassociated with variable 52. As is shown in Fig. 6, the SPE and T2

statistics of KPCA and KECA exceed their respective control limitsfrom the sample 161 but both of the two methods cannot detectthe fault after a period of time. The reason is that the control loopsact to compensate for the change and then the variables return totheir normal value. Besides, we cannot confirm the variable 52 incontribution plots of KPCA and KECA. However, form the monitor-ing results of WT-KECA in the third column, we can see theperformance of SPE and T2 statistics is significantly improved.Especially, the accurate fault variable (variable 52) can be identi-fied directly in the contribution rate chart. Therefore, the WT-KECAmethod further demonstrates its superiority in process monitoring.

5. Conclusion

In this paper, a fault detection method based on kernel entropycomponent analysis which produces strikingly different trans-formed data sets compared to KPCA is proposed. It proves thatKECA is a more efficient method for dynamic multivariate indus-trial process monitoring. At the same time, an improved faultdetection method combined the wavelet transform with KECA(WT-KECA) is also proposed. Extensive simulation results in TEprocess reveal that the KECA and WT-KECA methods are appro-priate approaches to detect faults.

References

[1] J.F. MacGregor, T. Kourti, Statistical process control of multivariate processes,Control Eng. Pract. 3 (1995) 403–414.

[2] Xiaoqiang Zhao, Xinming Wang, Wu Yang, An improved FVS-KPCA method offault detection on TE process, ICDMA (2012) 186–189.

[3] Bernhard Scholkopf, et al., Nonlinear component analysis as a kernel eigen-values problem, Neural Comput. 10 (1998) 1299–1319.

0 200 400 600 800 10000

0.5

1

1.5

KE

CA

-SP

E

sample

0 200 400 600 800 10000

5000

10000

15000

KE

CA

-T2

sample

0 10 20 30 40 50 600

2

4

6

8

KE

CA

-con

tribu

tion

rate

(%)

variable

0 200 400 600 800 10000

100

200

300

KP

CA

-SP

E

sample

0 200 400 600 800 10000

100

200

300

KP

CA

-T2

sample

0 10 20 30 40 50 600

2

4

6

8

KP

CA

-con

tribu

tion

rate

(%)

variable

0 200 400 600 800 10000

0.5

1

1.5

2

WT-

KE

CA

-SP

E

sample

0 200 400 600 800 10000

1

2

3

4x 104

WT-

KE

CA

-T2

sample

0 10 20 30 40 50 600

5

10

15

20

WT-

KE

CA

con

tribu

tion

rate

(%)

variable

Fig. 6. Monitoring charts of fault 5 for KPCA, KECA and WT-KECA.



http://refhub.elsevier.com/S0925-2312(14)00817-0/sbref1









[4] J.M. Lee, C.K. Yoo, S.W. Choi, P.A. Vanrolleghem, I.B. Lee, Nonlinear processmonitoring using kernel princ ipal component analysis, Chem. Eng. Sci. 59(2004) 223–234.

[5] R. Jenssen, Kernel entropy component analysis, IEEE Trans. Pattern Anal. Mach.Intell. 32 (2010) 847–860.

[6] Luis Gómez-Chova, Robert Jenssen, Gustavo Camps-Valls, Kernel entropycomponent analysis for remote sensing image clustering, IEEE Geosci. RemoteSens. Lett. 9 (2012) 312–316.

[7] B.H. Shekar, M. Sharmila Kumari, Leonid M. MestetskiyAuthor Vitae,F. Dyshkant Natalia, Face recognition using kernel entropy componentanalysis, Neurocomputing 74 (2011) 1053–1057.

[8] Zhibing Xie, Ling Guan, Multimodal information fusion of audio emotionrecognition based on kernel entropy component analysis, in: Proceedings ofthe IEEE International Symposium on Multimedia Date of Conference, 2012,pp. 1–8.

[9] Luis Gómez-Chova, Robert Jenssen, Gustavo Camps-Valls, Kernel entropycomponent analysis in remote sensing data clustering, Geosci. Remote Sens.Symp. (2011) 3728–3731.

[10] H. Monsef, S. Lotfifard, Internal fault current identification based on wavelettransform in power transformers, Electr. Power Syst. Res. 77 (2007)1637–1645.

[11] A. Renyi, On Measures of Entropy and Information, Sel. Pap. Alfred Renyi 2(1976) 565–580.

[12] Li Wen, Zheng shi Liu, Yun jian Ge, Several methods of wavelet denoising,J. Hefeiuniversity Technol. 25 (2002) 176 (172).

[13] Xiaoqiang Zhao, Xinming Wang, A fault detection algorithm based on waveletdenoising and KPCA, Adv. Intell. Soft Comput. 159 (2012) 311–317.

[14] P. Hu, A.-ping Yang, Indefinite kernel entropy component analysis, Sci.Technol. 3 (2010) 0–3.

[15] C. Soon Ong, X. Mary, S. Canu, A.J. Smola, Learning with non-positive kernels,in: Proceedings of International Conference on Machine Learning, 2004pp. 639–646.

[16] S. Belongie,C. Fowlkes,F. Chung, J. Malik, Spectral partitioning with indefinitekernels using the nystrom extension, in Proceedings of European Conferenceon Computer Vision, 2002, pp. 531–542.

[17] M.P. Wand, M.C. Jones, Kernel Smooting, fifth ed, Chapman and Hall, London,1995.

[18] Mohamed N. Nounou, Bhavik R. Bakshi, On-line multiscale filtering of randomand gross errors without process models, AIChE J. 45 (1999) 1041–1058.

[19] Xiaogang Deng, Xuemin Tian, Nonlinear process fault diagnosis method usingkernel principal component analysis, J. Shandong Univ. 35 (2005) 103–106.

Yinghua Yang received the B.S. degree from North-eastern University, Shenyang, China, in 1991 and theM.S. and Ph.D. degree in Control Theory and ControlEngineering from Northeastern University, Shenyang,China in 1994 and 2002 respectively. He is an associateprofessor in the College of Information Science andEngineering, Northeastern University in Shenyang. Hisresearch interests on process monitoring and faultdiagnosis based on statistical analysis theory.

Xiaole Li received bachelor degree in 2012 from InnerMongolia University of Technology, Hohhot, China. Heis currently a master degree candidate in NortheasternUniversity. His main research interests include processmonitoring and fault diagnosis based on statisticalanalysis theory.

Xiaozhi Liu received the B.S. degree from ShenyangUniversity of Technology, Shenyang, China, in 1990 andthe M.S. and Ph.D. degree in Control Theory and ControlEngineering from Northeastern University, Shenyang,China in 1995 and 2005 respectively. She is an associateprofessor in the College of Information Science andEngineering, Northeastern University in Shenyang. Herresearch interests on signal processing theory.

Xiaobo Chen received the B.S. degree from North-eastern University, Shenyang, China, in 1982 and theM.S. degree in Mechanical Engineering from North-eastern University, Shenyang, China in 1985. He servesas a professor in the College of Information Science andEngineering, Northeastern University in Shenyang. Hisresearch interests on process monitoring, fault diagno-sis and signal processing theory.





































Documents

Wavelet kernel entropy component analysis with application to industrial process monitoring