Blind equalization using a predictive radial basis function neural network

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 16, NO. 3, MAY 2005 709

Blind Equalization Using a Predictive Radial BasisFunction Neural Network

Nan Xie and Henry Leung, Member, IEEE

Abstract—In this paper, we propose a novel blind equalizationapproach based on radial basis function (RBF) neural networks.By exploiting the short-term predictability of the system input,a RBF neural net is used to predict the inverse filter output. Itis shown here that when the prediction error of the RBF neuralnet is minimized, the coefficients of the inverse system are iden-tical to those of the unknown system. To enhance the identifica-tion performance in noisy environments, the improved least square(ILS) method based on the concept of orthogonal distance to reducethe estimation bias caused by additive measurement noise is pro-posed here to perform the training. The convergence rate of theILS learning is analyzed, and the asymptotic mean square error(MSE) of the proposed predictive RBF identification method is de-rived theoretically. Monte Carlo simulations show that the pro-posed method is effective for blind system identification. The newblind technique is then applied to two practical applications: equal-ization of real-life radar sea clutter collected at the east coast ofCanada and deconvolution of real speech signals. In both cases,the proposed blind equalization technique is found to perform sat-isfactory even when the channel effects and measurement noise arestrong.

Index Terms—Autoregressive (AR) system, blind equalization,chaos, nonlinear prediction, radar, radial basis function (RBF)neural network, speech signal, system identification.

I. INTRODUCTION

THE PROBLEM of blind equalization or blind system iden-tification is to estimate the unknown system parameters

and structure based on the system output only without using theinput signal. Blind equalization finds applications in many areasincluding communications, control and signal processing. Forinstance, in a communication system, the transmitted signal isoften corrupted by channel interference, and blind equalizationis applied to eliminate the channel distortion and multipath ef-fects since the transmitted signal is unknown at the receiver end.

Assuming that the system to be identified can be described byan AR model, the identification problem can be formulated as

(1)

where is the order of the AR model,are the system parameters to be identified, is the systeminput or the driven signal, is the output signal, and

. The notationrepresents matrix transpose. The received signal is usually

Manuscript received September 26, 2003; revised September 10, 2004.The authors are with the Department of Electrical and Computer Engineering,

University of Calgary, Calgary, AB T2N 1N4, Canada (e-mail: [email protected];[email protected]).

Digital Object Identifier 10.1109/TNN.2005.845145

contaminated by a zero mean independent white Gaussian noiseprocess with variance , that is

(2)

The goal of blind identification is to estimate based on the re-ceived measurement only. It should be noted that the ARmodel is employed in the present study since an AR model canrepresent any linear system according to the Wold decomposi-tion [1]. In fact, it is widely used in many practical channel andsystem applications such as mobile communications and indoorradio [2], [3].

Many techniques have been developed for blind equaliza-tion in the literature. Among them, the most conventional ap-proach is the higher order statistics (HOS) [4], [5]. However,the HOS-based methods are computational heavy since they in-volve the computation of higher order moments. In addition,they are slow in convergence and require more samples to reachthe convergence. These weaknesses limit the HOS-based blindmethods in practical use. To overcome these problems, Gardner[6] and Tong [7] propose using second-order statistics (SOS)for blind channel equalization. The advantage of the SOS-basedmethods is that they have a faster convergence but they fail inidentifying the nonminimum phase and singular systems. Re-cently due to the success in applying neural networks to theblind source separation problem [8]–[12], blind equalizationbased on neural networks has also drawn a lot of attention.These methods perform blind equalization by exploiting dif-ferent characteristics of neural networks. Some researchers usethe impressive classification capability of neural nets to performequalization [13], while some formulate blind equalization asan optimization problem and use a recurrent neural network toachieve faster convergence [14]. In [15], the multilayer structureof a linear network is used for whitening and estimation to per-form blind identification. But in most cases [8], [16], [17], it isthe universal approximation capability of neural networks thatmakes the neural blind equalization approaches outperform theconventional methods.

In this paper, we propose a novel neural blind equalizationapproach based on the superior predictive ability of neural net-works. The idea is based on a recently proposed equalizationconcept called the minimum nonlinear prediction error (MNPE)method [18]. Assuming the system input signal is generatedby a nonlinear system where

and is the embedding dimen-sion of the input signal, the MNPE method passes the systemoutput to an inverse filter which is a standard approach in equal-izing the system effect. By minimizing the prediction error ofthe inverse filter output based on the nonlinear mapping

1045-9227/$20.00 © 2005 IEEE

710 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 16, NO. 3, MAY 2005

Fig. 1. Block diagram of the MNPE blind system identification. The driven signal is generated by some chaotic dynamics f( � ). The channel equalizer is consistedof an inverse filter and a nonlinear predictor, and the nonlinear predictor may contain a RBF neural network.

with respect to the inverse filter coefficients, it is shown that theinverse filter becomes an exact inverse of the unknown systemand the system can be identified. Compared to the statisticallyoptimal least square (LS) identification with white Gaussiandriven signals, the MNPE approach is shown to achieve superioridentification performance when the signal-to-noise ratio (SNR)is high [18].

The assumption of system input nonlinearity may sound re-strictive. But recent development in the theory of chaos andnonlinear dynamics show that a great variety of real-life sig-nals such as radar, acoustic and biomedical signals are chaoticrather than purely random [19]–[23]. Therefore, blind equaliza-tion of system driven by signals generated by a nonlinear dy-namical system becomes practical in many applications such asdeconvolution of audio signal, hands-free telephone [24], [25]and channel equalization of chaos communication [26], [27].

The MNPE method in [18] assumes that the input signal dy-namics is known as a priori which is not necessary the case inmany applications. In many situations such as radar and speech,although these signals are shown to follow a chaos descriptionclosely but their exact underlying dynamics are usually not ac-cessible. To overcome this problem, neural networks are pro-posed here to develop a more effective blind predictive equaliza-tion method due to their impressive predictive power [28], [29].In particular, the radial basis function (RBF) net is employed inthe present study because of its regularization and real-time pro-cessing capacity. More precisely, a RBF net is used as a predictorto approximate the underlying nonlinearity of . That is

(3)

where is the RBF, and is the approximation error. Itis shown here that by using a RBF predictor, the requirement onknowing the underlying dynamics of can be removed. Infact, this neural net approach does not really need the assump-tion that the system input is chaotic. It can be applied to a widerange of signals that a neural network can predict.

Another drawback of the MNPE approach is its sensitivityto measurement noise. Since additive measurement noise is un-avoidable in many applications such as communication channelequalization, the noise performance of the MNPE technique hasto be improved. We propose using the improved least square(ILS) criterion developed in [30] and [31] to replace the LS esti-mator in the MNPE method. The proposed ILS criterion is basedon minimizing the orthogonal Euclidean distance between thenoisy filter output and the true signal and reduces the estimationbias caused by additive measurement noise. In fact, the proposed

adaptive ILS method can be treated as a new adaptive filter thatimproves the efficiency of the conventional adaptive LS filter inhandling in-variable errors [32].

To evaluate the practicality of the proposed method, the pre-dictive neural equalization method is applied to two applica-tions. The first one is blind equalization of real-life radar sig-nals. The radar signal used in the present study is sea cluttercollected by an X-band coherent radar system [33]. Sea clutterrefers to the electromagnetic wave backscatter from a sea sur-face. In practice, the radio-frequency electromagnetic pulsesare usually distorted by the response of the sea surface andthe recording system [34]. The second application is blind de-convolution of speech signals which is a major problem of ahands-free telephone In both applications, the proposed RBFpredictive blind equalization method is found to be effectiveeven when the channel effect and measurement noise are strong.

This paper is organized as follows. Section II briefly presentsthe MNPE method using a linear autoregressive (AR) systemfor channel modeling. In Section III, the proposed RBF-MNPEmethod is presented and the ILS technique is used for training toreduce the estimation biases caused by the measurement noise.Theoretical analyzes including mean square error (MSE) perfor-mance and the convergence rate of the proposed method are alsoderived. Section IV reports the Monte Carlo simulation results.In Section V, the two real-life applications of the RBF predictiveblind equalization method are reported. Conclusions are givenin Section VI.

II. BLIND IDENTIFICATION BASED ON THE CONCEPT OF MNPE

The following general assumptions are made throughout thesequel. First, the AR system is of finite order , stable and time-invariant. Second, the system order is assumed to be known asa priori.

Fig. 1 illustrates the basic idea of the blind identificationbased on the concept of the MNPE method. The inverse filteris used to reverse the system effect so that ideally the filteroutput is equal to the original input signal, i.e., .Assuming that comes from a nonlinear system givenby , the MNPE method uses the con-cept that if is indeed equal to as desired, thencan be substituted into the nonlinear system for to have

.In other words, if is equal to , then we havean ideal inverse system in Fig. 1 that produces an output

equivalent to the system input . The MNPE approachtherefore optimizes the nonlinear prediction error (NPE) of the

XIE AND LEUNG: BLIND EQUALIZATION USING A PREDICTIVE RBF NEURAL NETWORK 711

inverse filter output with respect to the inverse systemcoefficients [18]. That is

(4)

where is the expectation operation, the inverse filter pa-rameters are estimated by using the LSmethod in [18], and the inverse filter output is given by

. It is proved in [18] that the necessary andsufficient condition for (4) to reach the minimum is .

In the present study, accuracy of parameter estimation is mea-sured by MSE given by

MSE (5)

where is normalized by the true system parameter. The SNR is defined as SNR where repre-

sents the average power of , that is, .

III. RBF-MNPE BLIND EQUALIZATION METHOD

A. Nonlinear System Approximation Using RBF NeuralNetworks

The assumption of knowing the exact dynamical function ofthe driven signal might not be realistic in many applications.Sinceneuralnetworksareuniversalapproximator,aRBFnetworkis used to replace in the predictive objective function in (4). Itshould be noted that the RBF net is used here because of its goodfunction approximation capability and fast learning rate. Othernetworks such as the conventional multilayer perceptron (MLP)can also be used but its slow convergence rate makes it inappro-priate for the real-time blind equalization applications.

A -input and single output RBF network as shown in Fig. 2 isused here. It consists of three layers: input layer, hidden layer, andoutput layer. The neurons in hidden layer called the RBF centersare of local response to its input and the neuron in the output layeronly sums up the outputs of the hidden neurons. Mathematically

(6)

where the Euclidean norm is used, ’s are the weights whichconnect the hidden center units to the network output,is the th RBF center and is its width. We estimate the stan-dard deviation using the k-nearest neighbors heuristic, i.e.,

, where is chosen as 2here. This method has the advantage of taking the signal distri-bution into account [32]. The RBF centers are obtained by usingK-means clustering [35] and fixed input embedding dimension.That is, the initial centers are randomly chosen from the trainingset, and the number of centers is fixed in the training procedure.For each input pattern , compare the Euclidean distanceand find the closest RBF center, then move the center slightlytoward the input pattern, i.e., ,where is the th RBF center at time instant , the update rate

is chosen as 0.1. The output layer weights are determined byLS so that this approach can be used for real-time processing. Inthe present study, the Gaussian RBF is used but the method canbe applied by using other RBFs. Compared to the MLP neural

Fig. 2. RBF networks with d-inputs and one output.

network, the backpropagation algorithm solves a nonlinear ap-proximation problem by a single learning phase, while the RBFtraining procedure proposed here is a 2-phase learning algorithm,that is, K-means for the RBF layer and LS-optimization for theoutput layer. This pretrained RBF net could be further tunedthrough a third training phase, where all parameters are trainedsimultaneously. And this so-called 3-phase RBF learning can beused to solve the general nonlinear approximation problem. Itshould also be noted that in some practical situations, maynot be available for training the RBF net. But signals with similardynamics can be used for training the neural net. For instance, inradar target detection in clutter, the exact clutter process in therange cell for detection is usually not available, but a predictiveclutter model can be built using clutter data from the nearbyrange cells. The details of the radar application will be given inSection V.

B. RBF-MNPE Method

Using a RBF net in the NPE objective function in (4), we have

(7)

The gradient descent method is usually applied in blind equal-ization [36], [37]. When it is applied to the previous NPE objec-tive function, we have

(8)

where is the step size.The previous formulation is based on the LS method that min-

imizes the mean square regression error for the inverse filteroutput with respect to the delay vector . But when

is corrupted by noise, the output signal of the nonlinear pre-dictor will be biased from the ideal desired signal

. This usually results in a biased estimation as shown in [30]and [32]. To handle this measurement noise problem, the fol-lowing ILS-NPE objective function is proposed:

(9)


The ILS minimizes the orthogonal Euclidean distance be-tween the noisy point and the surface defined by

. Note that the objective function in (9) is rather diffi-cult to use because of the minimization operator. The followingapproximation given in the following is found to be more prac-tical [31]:

(10)

where anddenotes a partial derivative over . Therefore, the ILSobjective function can be expressed as

(11)

Following the same derivation in (8), the gradient search tech-nique can be applied to minimize the proposed ILS-NPE as

(12)

which leads to the estimate of .

C. MSE Performance Analysis

To understand the effectiveness of the proposed method, the-oretical MSE analysis is carried out in this section. We first as-sume that is known and derive the MSE performance. Theresult is then extended to the situation that is unknown.

If the measurement noise 0, it is proved in [18] thatwhen (4) is minimized. Thus, the MSE is equal to zero in

this special case. But in general, . Due to the complexityof multidimensional chaos, we assume that is generatedby an one-dimensional (1-D) system in the following analysis,that is, . For multidimensional system, theapproximation error brought by using a 1-D system should bethe same for both LS-MNPE and ILS-MNPE methods, thereforethe conclusion from the present analysis is expected to hold. Butthe specific improvement resulted by using ILS-MNPE methodmay need to be justified for a multidimensional chaos system inthe further study.

The inverse filter output is expressed as

(13)

where .Let , since

, we have

(14)

For the LS-MNPE method, system parameters are es-timated by minimizing the NPE in (4). Using Taylor’sexpansion, can be expanded at and,hence, , whereis the first-order derivative with respect to . Notethat for a multidimensional system, and

, where the dimension of andis greater than one. According to (14), we can still

expand at using Taylor’s expansion. Andthe analysis will be similar to that of a 1-D system.

Combined with (14), the LS-NPE becomes

(15)

Note that these equations are valid only with low noise. Since, let represent

the covariance of . It could beshown that

(16)

Substituting (16) into (15), the LS-NPE can be expressed as

(17)

Minimizing the LS-NPE with respect to , we have

(18)

Therefore

(19)


The optimal is given by

(20)

The corresponding MSE of the LS-MNPE method using theexact nonlinear function can then be expressed as

MSE (21)

Note that in (21), MSE 0 when 0. This is consistentwith the previous conclusion for a noiseless system.

For the RBF-MNPE method, the driven signal is predicted byusing a RBF net. According to (3), (14) and Taylor’s expansion,we have

, where represents the first-order derivative.Let , the NPE of the LS-RBF-MNPEmethod becomes

(22)

Since , we have

(23)

Since the approximation error is assumed to be a zero-mean independent process, it is orthogonal to , and (23)can be further expanded. That is

(24)Therefore

(25)

Minimizing (25) with respect to , we have

(26)

Therefore, the optimal value is equal to

and the MSE of the LS-RBF-MNPE can be expressed as

MSE (27)

The final MSE study is to extend the previous analysis to theILS estimation, note that

and, hence, the ILS-NPE in (9) can beexpressed as

(28)

According to (16),. It follows that:

(29)

Minimizing (29) with respect to , we have

(30)

Therefore, the optimal system parameters are given by, and the asymptotic MSE of the ILS-

MNPE identification method is equal to

MSE (31)

For the RBF-MNPE method using ILS, the NPE becomes

(32)

Since, let , where the

approximation error is a zero-mean independent process,we have

(33)

Substituting into the ILS-NPE in (32), it follows that:

(34)

Using (16) and minimize (34) with respect to , we have

(35)


Therefore, the optimal value can be obtained as

and the corresponding MSE can be expressed as shown in (36)at the bottom of the page.

D. Convergence Analysis

In addition to the MSE performance, convergence is anotherimportant issue that has to be addressed for real-time equaliza-tion. For the sake of clear presentation, let and

. For the RBF-MNPE method to be carried out in anadaptive fashion for real-time applications, the updated equa-tion for can be expressed as

(37)

where . Subtracting from both sides of (37),we have

Let , we have

(38)

Similarly, the gradient update equation of the ILS-RBF-MNPEmethod can be expressed as

(39)

where and .Subtracting from both sides of the previous equation, we have

Let

(40)

Comparing (38) with (40), it is noted that the convergencerate of the LS-MNPE and ILS-MNPE methods are determinedby and , respectively. Assuming that the RBF neuralnetwork can approximate the nonlinear map accurately, i.e.,

, we have and . Hence, (40) becomes. Since , we have .

In addition, because and, it follows that the convergence rate of ILS-RBF-

MNPE is in general slower than that of LS-RBF-MNPE [38].Furthermore, for stable learning, the step size must be in therange for the LS-RBF-MNPE andfor the ILS-RBF-MNPE, respectively. Since , wehave . In other words, the ILS-RBF-MNPE has a wider range of for a converging adaptation.

IV. COMPUTER SIMULATION

The widely used logistic map

(41)

is used as the system input in the sequel. We set 4.0 sothat is chaotic and uncorrelated. Using k-means clustering,a RBF net with an input embedding dimension of unity and dif-ferent number of centers from one to twenty is used to approx-imate the logistic map signal. The number of training points is1000 and the size of testing set is 500. It is found that using tenhidden centers can achieve the best approximation performance.The prediction MSE is about 120 dB. As shown in Fig. 3(a)the approximation is very close to the true dynamics. Based onthe trained RBF net, the proposed RBF-MNPE method is usedto identify a linear AR model of order two with coefficients [39]

(42)

MSE (36)


Fig. 3. (a) Comparison of the true delay plot and the approximate delay plotfrom a RBF network. The chaotic signal is a logistic map. (b). Comparison ofthe logistic map true delay plot and the approximate delay plot from a MLPnetwork.

To have a clear understanding of the performance, we first con-sider the case of no measurement noise.

The simulation is repeated using a three layer MLP neuralnet. The input embedding dimension is fixed as one and thenumber of hidden layer perceptron varies from five to twenty.Here we use the standard backpropagation algorithm [35] totrain the MLP. Compared to RBF net, the convergence of MLPis more than 10 times slower. It is found that the MLP with morethan five hidden layer perceptions could not result in any signifi-cant improvement in the prediction performance. Therefore, thenumber of hidden neurons is fixed as five. The prediction MSEis found to equal to 115 dB, and the predicted phase spaceportrait is shown in Fig. 3(b).

Fig. 4 is plotted the equalization performance of (42) versusdifferent numbers of samples. The MSE performance is aver-aged over 100 independent trials. The step size is chosen bytrial and error. For the RBF-MNPE method, we have 0.2,and it gives the best performance after 128 points. The MNPE

Fig. 4. MSE performance versus number of points in the received signal. Twodriven signals are considered: a logistic map and white Gaussian driven signal.The RBF-MNPE and the MLP-MNPE methods are also compared to the MNPEmethod using the true chaotic dynamic.

Fig. 5. Comparison of noise performance between LS-RBF-MNPE methodusing chaos and LS method using white Gaussian driven signal. A logistic mapdriven signal is used in the MNPE method.

method using the true logistic map is also considered for com-parison. Its step size is chosen as 0.1. It is found that theirperformances are very close after 128 points. This indicates thatthe RBF approximation is very accurate. Using MLP, the MNPEmethod uses a learning rate of 0.015. It achieves the best MSEafter 256 points.

Next, we compare the RBF-MNPE method using chaos withthe statistically optimal LS method with white Gaussian drivensignals. That is, is a zero mean white Gaussian process withunit variance. Since LS with white Gaussian signals is a statisti-cally optimal approach, it is a good benchmark for the proposedequalization method. The comparison is shown in Fig. 5. It canbe seen that the RBF-MNPE method significantly outperformsthe LS method with white Gaussian signals. The improvementis up to 50 dB.

Convergence of the RBF-MNPE method is illustrated inFig. 6. The results are plotted over an average of 100 inde-


Fig. 6. Comparison of the convergence time between the ILS-RBF-MNPE andLS-RBF-MNPE methods for an AR model h = [0:195; 0:95]. SNR = 10 dB.

Fig. 7. Comparison of parameter estimation MSE versus SNR between theILS-RBF-MNPE and LS-RBF-MNPE techniques. The driven signal is a logisticmap.

pendent trials. 1000 points are used in the convergence test.It is interesting to point out that the LS-RBF-MNPE methoddiverges for 0.1 while the ILS approach still converges.This confirms our theoretical analysis that the ILS-RBF-MNPEmethod has a wider stable range for the step size . AtSNR 10 dB, both ILS and LS algorithms converge, and theILS approach converges to a more accurate estimate than thatof the LS method. The LS-RBF-MNPE, although convergesslightly faster, locates its final value relatively far away fromthe true parameters.

Fig. 7 compares the performance of the ILS-RBF-MNPE andLS-RBF-MNPE methods in identifying an AR(4) system withcoefficients

(43)

The results are averaged over 100 independent trials after thealgorithms are converged. 1000 points are used in each trial.The step size is chosen as 0.05 and 0.01 for the ILS-RBF-MNPE and LS-RBF-MNPE methods, respectively. It is shown

Fig. 8. Comparison of MSE between the RBF-MNPE methods and the MNPEtechnique using exact driven signal dynamic. The driven signal is a logistic map.

Fig. 9. Prediction performance of the sea clutter signal at range 2200 m usinga RBF neural network. (a). The RBF net is trained using clutter signal at range2200 m. The prediction MSE is �23.94 dB. (b). The RBF net is trained usingclutter signal at range 2400 m. The prediction MSE is �21.39 dB.


Fig. 10. Comparison of the recovered sea clutter image at SNR = 25 dB. (a). Original sea clutter image. (b). Recovered signal without channel equalization.RMSE = �21.95 dB. (c). Recovered signal using standard LS method. RMSE = �33.01 dB. (d). Recovered signal using the ILS RBF-MNPE equalizationmethod. RMSE = �43.85 dB.

that the ILS-RBF-MNPE method consistently outperforms theLS-RBF-MNPE approach. The improvement is about 2–4 dBdepending on the SNR. When SNR is high, the performanceof these two approaches are close to each other. This is notsurprising because when the measurement noise ,both criteria become the same. This observation can also beconfirmed from the asymptotic MSE analysis in (27) and (36).Fig. 8 compares the RBF-MNPE methods to the MNPE ap-proach using the exact nonlinear map in identifying the ARsystem (43) with different SNRs. It is observed that the perfor-mances between assuming the knowledge of and using an ap-proximation are quite close. This demonstrates the effectivenessof using neural nets in this blind equalization application.

V. APPLICATIONS

A. Blind Equalization of Radar Signals

In this study, the proposed RBF predictive blind equalizationtechnique is applied to equalize real radar signal collected byan X-band coherent radar system. The radar is a coherent, dual-polarized system, at a site at Osborne Head Gunnery Range,Nova Scotia, Canada, in November 1993. The staring data setwas obtained by aiming the radar dish to the Atlantic Ocean ata single azimuth for various range bins and the received radar

return is the backscattered signal from the sea called sea clutter.The details of the operating condition of the radar system anddata characteristics are given in [33] and [40].

In many situations, electromagnetic signal might sufferfrom multipath and other channel effects, and in this casethe received signal becomes the convolution of the sea sur-face’s response and the recording system response. In orderto restore a pure clutter signal, we can apply the proposedneural blind equalization method to cancel these channeleffects. In order to assess the equalization performance, weassume that the channel interference is modeled by a time-in-variant linear system. An AR(8) model with coefficient vector

isused in our experiment. In fact, we also performed similarexperiments on AR models with different orders (from twoto ten), and have the same observation. In order to have aclearer interpretation of the results, the sea clutter signal is firstnormalized to before any processing. Both predictionand equalization MSE performance are calculated based on thenormalized signals.

In the following simulations, all the results are obtained afterthe transient period. Different input embedding dimensions andhidden centers are used to train the RBF predictor. The detailedparameter setting of the RBF net can be found in [40]. The first


1000 points of the clutter signal at range 2200 m are used totrain the RBF net. It is found that the RBF net with an embed-ding dimension of 10 and 70 hidden centers achieves the bestprediction performance. The prediction MSE is 23.94 dB. Theprediction performance is demonstrated in Fig. 9(a) where theprediction is quite close to the actual value.

In the previous simulation, the RBF net is trained using cluttersignal at the range of 2200 m, and it is used to predict thesignal at the same range but different azimuth. It should be notedthat in some cases the RBF predictor has to be trained usingclutter signal from other range cells. However, the underlyingdynamics of the clutter signal at a specific range is usually verysimilar to those at the adjacent ranges [40]. To illustrate thispoint, we use the first 1000 points of the sea clutter signal at therange of 2400 m to train a RBF net with an embedding dimen-sion of 10 and 70 hidden centers. This RBF net is then appliedto predict the clutter signal at the range of 2200 m. The predic-tion performance is shown in Fig. 9(b), and the prediction MSEis 21.39 dB. It shows that the training signal can be obtainedfrom a relatively large neighborhood which makes the proposedmethod more practical in this radar application.

The equalization performance is measured by the signal re-covery mean square error (RMSE)

RMSE dB (44)

where is the staring sea clutter signal andis the recovered information. Fig. 10(a) denotes the corre-

sponding radar image. The color map is 8-b and is in the rangeof [0, 1], that is, the image is in 256 shades of grayscale. Theclutter data set is in a matrix format, the elements of the clutterdata matrix are used as indices into the 8-b color map to deter-mine the color in the image. The axis in Fig. 10 denotes thetime and the axis represents the range. Due to the channeleffects, the received image is severely distorted as shown inFig. 10(b) where many useful information (for example, the darkhorizontal line which usually represents the small target in thesea) is lost. The RMSE is 21.95 dB. Using the RBF predictor,the ILS-RBF-MNPE method is applied to cancel the channeleffects. The step size is chosen as 0.015. One thousand datapoints are used for the gradient update. We also compare theperformance to that of the standard LS method, because bothdo not require any a priori information about the desired signal.Fig. 10(c) is for the recovered image using the LS method. TheRMSE is about 33.01 dB. Fig. 10(d) is plotted for the re-covered image using the ILS-RBF-MNPE channel equalizationmethod. The RMSE is about 43.85 dB. It is found that theRBF-MNPE method produces a reasonably good quality image.

B. Blind Deconvolution of Speech Signal

Existing speech enhancement techniques, such as interferencesubtraction, comb filtering, and speech resynthesis, are designedto remove additive noise [41]. However, another equally impor-tant category of degradation sources is the convolutional noise[24]. These distortions are caused by the acoustical properties oftheenvironment, theconversionmediaor the transmissionmedia.

Fig. 11. (a) Waveform of the speech signal “where is the kaboom.” (b)Prediction performance of the speech signal using a RBF net.

Fig. 12. Comparison of the channel parameter estimation MSE for thespeech signal deconvolution using the LS-RBF-MNPE and ILS-RBF-MNPEtechniques.

Recently, it has been shown that speech signals could be modeledby dynamical attractors in a relatively low-dimensional phase


Fig. 13. (a). Part of the recovered speech signal using the standard LSmethod. RMSE = �5.89 dB. (b). Part of the recovered speech signal usingthe LS-RBF-MNPE method. RMSE = �12.14 dB. (c). Part of the recoveredspeech signal using the ILS-RBF-MNPE method. RMSE = �14.92 dB.

space. But in practice, the underlying dynamics of the speechsignal is unknown and, hence, the MNPE method is not

applicable. Hereby we can apply the LS-RBF-MNPE andILS-RBF-MNPE methods to improve the speech signal recoveryquality from convolutional noise.

A speech signal recorded from a male’s voice saying “whereis the kaboom” is used for demonstration. The original signalis shown in Fig. 11(a). From trials with different embeddingdimension from 2 to 10, a RBF net with an embedding dimensionof 7 is found to provide the best prediction performance. Usingk-means clustering, 70 RBF hidden centers are employed tomodel the speech signal. The number of training points is 1000,and the validation length is 500. Fig. 11(b) shows its predictionperformance. It is found that the prediction MSE is equal to

22.95 dB. When the LS-RBF-MNPE and the ILS-RBF-MNPEmethods are applied to equalize the speech signal, a step sizeof 0.1 is chosen for both ILS-RBF-MNPE and LS-RBF-MNPEmethods. The effect of the linear convolutional noise source issimulated by the AR(8) system used previously. The parameterMSE is plotted In Fig. 12. The ILS-RBF-MNPE consistentlyoutperforms the LS-RBF-MNPE, and the improvement is from1–3 dB. An intuitive comparison of the recovered speech signalusing different equalization methods at SNR 30 dB is shownin Fig. 13. As we can see, the ILS-RBF-MNPE method has animprovement of 2 dB compared to the LS-RBF-MNPE, and itsMSE is about 9 dB better than that of the standard LS method.

VI. CONCLUSION

In this paper, we propose a novel neural predictive blindequalization approach to identify linear systems in a noisyenvironment. By using a RBF neural net to predict the inversesystem output, it is shown here that the unknown system canbe identified when the prediction error of the inverse filteroutput is minimized. The ILS criterion is developed to reducethe estimation bias caused by additive measurement noise,and its convergence rate is also analyzed. The MSE of theproposed RBF predictive blind equalization method is derivedtheoretically, and the performance is confirmed by computersimulations. It is shown that the proposed method consistentlyoutperforms the conventional method. The effectiveness ofthe proposed method is further demonstrated by applying itto equalization of real radar sea clutter and deconvolution ofspeech signals. In both cases, the RBF predictive neural blindequalization method is found to have satisfactory performance.

REFERENCES

[1] H. Wold, A Study in the Analysis of Stationary Time Series. Uppsala,Germany: Almqvist and Wiksell, 1938.

[2] S. J. Howard and K. Pahlavan, “Autoregressive modeling of wide-bandindoor radio propagation,” IEEE Trans. Commun., vol. 40, no. 9, pp.1540–1552, Sep. 1997.

[3] T. Eyceoz, A. Duel-Hallen, and H. Hallen, “Deterministic channel mod-eling and long range prediction of fast fading mobile radio channels,”IEEE Commun. Lett., vol. 2, no. 9, pp. 254–256, Sep. 1998.

[4] C. L. Nikias and A. P. Petropulu, Higher-Order Spectra Analysis. En-glewood Cliffs, NJ: Prentice-Hall, 1993.

[5] G. B. Giannakis, Y. Hua, P. Stocia, and L. Tong, Signal ProcessingAdvances in Wireless and Mobile Communications, vol. 1: Trend inChannel Estimation and Equalization. Englewood Cliffs, NJ: Pren-tice-Hall, 2001.

[6] W. A. Gardner, “A new method of channel identification,” IEEE Trans.Commun., vol. 39, no. 6, pp. 813–817, Jun. 1991.


[7] L. Tong, G. Xu, and T. Kailath, “A new approach to blind identifica-tion and equalization of multipath channels,” in Proc. 25th AsilomarConf. Signals, Systems and Computers, Pacific Grove, CA, Nov. 1991,pp. 856–860.

[8] Y. Tan, J. Wang, and J. M. Zurada, “Nonlinear blind source separationusing a radial basis function network,” IEEE Trans. Neural Netw., vol.12, no. 1, pp. 124–134, Jan. 2001.

[9] A. Uncini and F. Piazza, “Blind signal processing by complex domainadaptive spline neural networks,” IEEE Trans. Neural Netw., vol. 14, no.2, pp. 399–412, Mar. 2003.

[10] S. Choi and T. W. Lee, “A negentropy minimization approach to adaptiveequalization for digital communication systems,” IEEE Trans. NeuralNetw., vol. 15, no. 4, pp. 928–936, Jul. 2004.

[11] E. Soria-Olivas, J. D. Martin-Guerrero, G. Camps-Valls, A. J. Serrano-Lopez, J. Calpe-Maravilla, and L. Gomez-Chova, “A low-complexityfuzzy activation fuzzy activation function for artificial neural networks,”IEEE Trans. Neural Netw., vol. 14, no. 6, pp. 1576–1579, Nov. 2003.

[12] D. Anguita, A. Boni, and S. Ridella, “A digital architecture for supportvector machines: Theory, algorithm, and FPGA implementation,” IEEETrans. Neural Netw., vol. 14, no. 5, pp. 993–1009, Sep. 2003.

[13] S. Chen, B. Mulgrew, and P. M. Grant, “A clustering technique for dig-ital communications channel equalization using radial basis functionnetworks,” IEEE Trans. Neural Netw., vol. 4, no. 4, pp. 570–590, Jul.1993.

[14] G. Kechriotis, E. Zervas, and E. S. Manolakos, “Using recurrent neuralnetworks for adaptive communication channel equalization,” IEEETrans. Neural Netw., vol. 15, no. 2, pp. 267–278, Mar. 1994.

[15] Y. Fang and T. W. S. Chow, “Blind equalization of a noisy channel bylinear neural network,” IEEE Trans. Neural Netw., vol. 10, no. 4, pp.918–924, Jul. 1999.

[16] S. Abrar, A. Zerguine, and M. Betlayeb, “Recursive least-squares back-propagation algorithm for stop-and-go decision-directed blind equaliza-tion,” IEEE Trans. Neural Netw., vol. 13, no. 6, pp. 1472–1481, Nov.2002.

[17] C. You and D. Hong, “Nonlinear blind equalization scheme usingcomplex-valued multilayer feedforward neural networks,” IEEE Trans.Neural Netw., vol. 9, no. 6, pp. 1442–1455, Nov. 1998.

[18] Z. Zhu and H. Leung, “Identification of linear systems driven by chaoticsignals using nonlinear prediction,” IEEE Trans. Circuits Syst. I, vol. 49,no. 2, pp. 170–180, Feb. 2002.

[19] H. Leung and T. Lo, “Chaotic radar signal processing over the sea,” IEEEJ. Ocean. Eng., vol. 18, pp. 287–295, Jul. 1993.

[20] A. J. Palmer, R. A. Kropfli, and C. W. Fairall, “Signatures of determin-istic chaos in radar sea clutter and ocean surfaces winds,” Chaos, vol. 5,pp. 613–616, 1995.

[21] A. Kumar and S. Mullick, “Nonlinear dynamical analysis of speech,” J.Acoust. Soc. Amer., vol. 100, pp. 615–629, Jul. 1996.

[22] M. Banbrook and S. McLaughlin, “Is speech chaotic?: Invariant geo-metrical measures for speech data,” in Proc. Inst. Elect. Eng. Colloq.Exploiting Chaos in Signal Processing, London, U.K., Jun. 1994.

[23] B. H. Jansen and M. E. Brandt, Nonlinear Dynamical Analysis of theEEG, B. H. Jansen and M. E. Brandt, Eds, Singapore: World Scientific,1993.

[24] S. Subramaniam, A. P. Petropulu, and C. Wendt, “Cepstrum-baseddeconvolution for speech dereverberation,” IEEE Trans. Speech AudioProcess., vol. 4, no. 5, pp. 392–396, Sep. 1996.

[25] A. M. Chan and H. Leung, “Equalization of speech and audio signalsusing a nonlinear dynamical approach,” IEEE Trans. Speech and AudioProcessing, vol. 7, no. 3, pp. 356–360, May 1999.

[26] H. Leung, “System identification using chaos with application to equal-ization of a chaotic modulation system,” IEEE Trans. Circuits Syst. I,vol. 45, no. 3, pp. 314–320, Mar. 1998.

[27] M. Ciftci and D. B. Williams, “Channel equalization for multiuserchaotic communications systems,” in Proc. IEEE Int. Conf. Acous-tics Speech Signal Processing, vol. II, Orlando, FL, May 2002, pp.1113–1116.

[28] H. Casdagli and S. Eubank, Nonlinear Modeling and Fore-casting. Reading, MA: Addison-Wesley, 1992.

[29] A. S. Weigend and N. A. Gershenfield, Time Series Prediction: Fore-casting the Future and Understanding the Past. Reading, MA: Ad-dison-Wesley, 1994.

[30] L. Jaeger and H. Kantz, “Effective deterministic models for chaotic dy-namics perturbed by noise,” Phys. Rev. E, vol. 55, pp. 5234–5247, May1997.

[31] Z. Zhu and H. Leung, “Adaptive identification of nonlinear systems withapplication to chaotic communications,” IEEE Trans. Circuits Syst. I.,vol. 47, no. 7, pp. 1072–1080, Jul. 2000.

[32] W. A. Fuller, Measurement Error Models. New York: Wiley, 1987.[33] A. Drosopoulos, “Description of the OHGR database,” Defense Res. Es-

tablish. Ottawa, Tech. Note 94-14, Dec. 1994.[34] G. Turner, “Subsurface radar propagation deconvolution,” Geophysics,

vol. 59, no. 2, pp. 215–223, Feb. 1994.[35] S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd

ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.[36] A. Benveniste and M. Goursat, “Blind equalizers,” IEEE Trans.

Commun., vol. COM-32, no. 8, pp. 871–887, Aug. 1984.[37] G. Picchi and G. Prati, “Blind equalization and carrier recovery using

a ‘Stop-and-Go’ decision-directed algorithm,” IEEE Trans. Commun.,vol. COM-35, no. 8, pp. 877–887, Aug. 1987.

[38] S. V. Vaseghi, Advanced Digital Signal Processing and Noise Reduction,2nd ed. New York: Wiley, 2000.

[39] S. Haykin, Adaptive Filter Theory, 2nd ed. Englewood Cliffs, NJ:Prentice-Hall, 1991.

[40] N. Xie, H. Leung, and H. Chan, “A multiple model prediction approachfor sea clutter modeling,” IEEE Trans. Geosci. Remote Sens. E, vol. 41,no. 6, pp. 1491–1502, Jun. 2003.

[41] D. O’Shaughnessy, “Enhancing speech degraded by additive noise orinterfering speakers,” IEEE Commun. Mag., vol. 27, no. 2, pp. 46–52,Feb. 1989.

Nan Xie received the B.S. and M.S. degrees in applied physics from South ChinaUniversity of Technology, Guangzhou, China, in 1995 and 1998, respectively.He is presently working toward the doctorate degree in the field of digital signalprocessing at the University of Calgary, Calgary, AB, Canada.

His current research interests include wireless communications, data mining,intelligent signal processing, nonlinear dynamics, and system identification.

Henry Leung (M’90) received the Ph.D. degreein electrical and computer engineering from theMcMaster University, Hamilton, ON, Canada.

He is currently a Professor in the Department ofElectrical and Computer Engineering, University ofCalgary, Calgary, AB, Canada. Before that he waswith the Defence Research Establishment Ottawa,ON, Canada, where he was involved in the design ofautomated systems for air and maritime multisensorsurveillance. His research interests include chaos,computational intelligence, data mining, nonlinear

signal processing, multimedia, radar, sensor fusion, and wireless communica-tions.

Documents

Blind equalization using a predictive radial basis function neural network