17
Neurocomputing 48 (2002) 339–355 www.elsevier.com/locate/neucom Parametric approach to blind deconvolution of nonlinear channels Jordi Sole a , Christian Jutten b; , Anisse Taleb b; c a University of Vic, Sagrada Fam lia 7, 08500, Vic, Spain b INPG, LIS, 46 Avenue F elix Viallet, 38031 Grenoble, Cedex, France c Ericsson Research, Ericsson Radio Systems AB, SE-164 80 Stockholm, Sweden Received 17 November 2000; accepted 6 June 2001 Abstract A parametric procedure for blind inversion of nonlinear channels is proposed, based on a recent method of blind source separation in nonlinear mixtures. Two parametric models are developed: a polynomial model and a neural model. The method, based on the min- imization of the output mutual information, needs the knowledge of log-derivative of the input distribution (score function). Each algorithm consists of three adaptive blocks: one for estimating the score function, and two other blocks for estimating the inverses of the linear and nonlinear parts of the channel. Experiments show that the algorithms perform eciently, even for hard nonlinear distortion. c 2002 Elsevier Science B.V. All rights reserved. Keywords: Mutual information; Independence; Wiener system; Blind deconvolution; Source separation 1. Introduction When linear models fail, nonlinear models appear to be powerful tools for mod- eling practical situations. Many researches have been done in the identication and= or the inversion of nonlinear systems. These works assume that both the input and the output of the distortion are available [13]; they are based on higher-order input= output cross-correlation [3], bispectrum estimation [11,12] or on the appli- cation of the Bussgang and Prices theorems [4,10] for nonlinear systems with Corresponding author. Tel.: +33-4-76-57-43-51; fax: +33-4-76-57-47-90. E-mail address: [email protected] (C. Jutten). 0925-2312/02/$ - see front matter c 2002 Elsevier Science B.V. All rights reserved. PII: S0925-2312(01)00651-8

Parametric approach to blind deconvolution of nonlinear channels

Embed Size (px)

Citation preview

Neurocomputing 48 (2002) 339–355www.elsevier.com/locate/neucom

Parametric approach to blind deconvolution of nonlinearchannels

Jordi Solea , Christian Juttenb;∗, Anisse Talebb;caUniversity of Vic, Sagrada Fam��lia 7, 08500, Vic, Spain

bINPG, LIS, 46 Avenue F�elix Viallet, 38031 Grenoble, Cedex, FrancecEricsson Research, Ericsson Radio Systems AB, SE-164 80 Stockholm, Sweden

Received 17 November 2000; accepted 6 June 2001

Abstract

A parametric procedure for blind inversion of nonlinear channels is proposed, based ona recent method of blind source separation in nonlinear mixtures. Two parametric modelsare developed: a polynomial model and a neural model. The method, based on the min-imization of the output mutual information, needs the knowledge of log-derivative of theinput distribution (score function). Each algorithm consists of three adaptive blocks: onefor estimating the score function, and two other blocks for estimating the inverses of thelinear and nonlinear parts of the channel. Experiments show that the algorithms performe5ciently, even for hard nonlinear distortion. c© 2002 Elsevier Science B.V. All rightsreserved.

Keywords: Mutual information; Independence; Wiener system; Blind deconvolution; Sourceseparation

1. Introduction

When linear models fail, nonlinear models appear to be powerful tools for mod-eling practical situations. Many researches have been done in the identi;cationand=or the inversion of nonlinear systems. These works assume that both the inputand the output of the distortion are available [13]; they are based on higher-orderinput=output cross-correlation [3], bispectrum estimation [11,12] or on the appli-cation of the Bussgang and Prices theorems [4,10] for nonlinear systems with

∗ Corresponding author. Tel.: +33-4-76-57-43-51; fax: +33-4-76-57-47-90.E-mail address: [email protected] (C. Jutten).

0925-2312/02/$ - see front matter c© 2002 Elsevier Science B.V. All rights reserved.PII: S0925-2312(01)00651-8

340 J. Sole et al. / Neurocomputing 48 (2002) 339–355

Gaussian inputs. However, in a real world situations, one often does not have accessto the distortion input. In this case, blind identi;cation of the nonlinearity becomesthe only way to solve the problem. This paper is concerned by a particular class ofnonlinear systems, composed by a linear ;lter followed by a memoryless nonlineardistortion (Fig. 1, left). This class of nonlinear systems, also known as a Wienersystem, is a nice and mathematically attracting model, but also a realistic modelused in various areas, such as biology [8], industry [2], sociology and psychology(see also [9] and the references therein). Despite its interest, today, there does notexist fully blind procedure for inverting such systems.In this paper, we proposed a fully blind inversion method inspired of recent

advances in source separation of nonlinear mixtures. Although deconvolution canbe viewed as a single input=single output (SISO) source separation problem inconvolutive mixtures (which are consequently not cited in this paper), the cur-rent approach is actually very diHerent. It is mainly based on equivalence be-tween instantaneous post-nonlinear (pnl) mixtures and Wiener systems, provideda well-suited parameterization. The paper is organized as follows. The Wienermodel and its parameterization are described in Section 2. The cost function basedon statistical independence is derived in Section 3. The gradient of the cost func-tion with respect to the inversion structure parameters are detailed in Section 4,according two parametric (polynomial and neural) models. Section 5 contains afew simulation results. In Section 6, we discuss on what happen if the inputsignal is not independent and identically distributed (i.i.d.), before concluding inSection 7.

2. Model and assumptions

We suppose that the input of the system S={s(t)} is an unknown non-Gaussiani.i.d. process, and that subsystems h; f are a linear ;lter and a memoryless nonlinearfunction, respectively, both unknown and invertible. We would like to estimates(t) by only observing the system output. This implies the blind estimation of theinverse structure (Fig. 1, right), composed of similar subsystems: a memorylessnonlinear function g followed by a linear ;lter w. Such a system is known as aHammerstein system. Let s and e be the vectors of in;nite dimension, whose tthentries are s(t) or e(t), respectively. The unknown input–output transfer can bewritten as

e= f(Hs); (1)

Fig. 1. The unknown nonlinear convolution system (left) and the proposed inversion structure (right).

J. Sole et al. / Neurocomputing 48 (2002) 339–355 341

where

H=

· · · · · · · · · · · · · · ·· · · h(t + 1) h(t) h(t − 1) · · ·· · · h(t + 2) h(t + 1) h(t) · · ·· · · · · · · · · · · · · · ·

(2)

is an in;nite dimension Toeplitz matrix which represents the action of the ;lter h tothe signal s(t). The matrix H is non-singular provided that the ;lter h is invertible,i.e. satis;es h−1h(t)= hh−1(t)= �(t), where �(t) is the Dirac impulse. The in;nitedimension of vectors and matrix is due to the lack of assumption on the ;lterorder. If the ;lter h is a ;nite impulse response (FIR) ;lter of order Nh, the matrixdimension can be reduced to the size Nh. Practically, because in;nite-dimensionequations are not tractable, we have to choose a pertinent (;nite) value for Nh.

Eq. (1) corresponds to a pnl model [14]. This model has been recently studiedin nonlinear source separation, but only for a ;nite dimensional case. In fact, withthe above parameterization, the i.i.d. nature of s(t) implies the spatial independenceof the components of the in;nite vector s. Similarly, the output of the inversionstructure can be written y=Wx with x(t)= g(e(t)). Following [14,15] the inversesystem (g; w) can be estimated by minimizing the output mutual information, i.e.spatial independence of y which is equivalent to the i.i.d. nature of y(t).

3. Cost function

The mutual information of a random vector of dimension n, de;ned by

I(z) =n∑

i=1

H (zi)−H (zi; z2; : : : ; zn) (3)

can be extended to a vector of in;nite dimension, using the notion of entropy ratesof stationary stochastic processes [6]

I(Z) = limT→∞

12T + 1

{T∑

t=−T

H (z(t))−H (z(−T ); : : : ; z(T ))}

=H (z(�))−H (Z); (4)

where � is arbitrary due to the stationarity assumption. We can notice that I(Z)is always positive and vanishes iH z(t) is i.i.d. Since S is stationary, and h and ware time-invariant ;lters, then Y is stationary too, and I(Y ) is de;ned by

I(Y ) =H (y(�))−H (Y ): (5)

342 J. Sole et al. / Neurocomputing 48 (2002) 339–355

Using the Lemma 1 of [15], the last right term of Eq. (5) becomes

H (Y ) =H (X ) +12�

∫ 2�

0log

∣∣∣∣∣+∞∑

t=−∞w(t)e−jt�

∣∣∣∣∣ d�: (6)

Moreover, using x(t) = g(e(t)) and the stationarity of E = {e(t)}

H (X ) = limT→∞

12T + 1

{H (e(−T ); : : : ; e(T )) +

T∑t=−T

E[log g′(e(t))]

}

=H [E] + E[log g′(e(t))]: (7)

Combining (6) and (7) in (5) leads ;nally to

I(Y ) =H (y(�))− 12�

∫ 2�

0log

∣∣∣∣∣+∞∑

t=−∞w(t)e−jt�

∣∣∣∣∣ d�− E[log g′(e(�))]−H [E]:

(8)

4. Theoretical derivation of the inversion algorithm

For deriving the optimization algorithm, the derivatives of I(Y ) (8), with respectto the parameters of the linear part w and of the nonlinear function g, are required.

4.1. Linear subsystem

The linear subsystem is parameterized by the coe5cients of the ;lter w. Thederivative of I(Y ) with respect to the coe5cient w(t), corresponding to the tth lagof the ;lter w, is computed as follow (see [15] for details).The derivative of the ;rst right-side term of (8) is

@H (y(�))@w(t)

=−E[p′

Y (y(�))pY (y(�))

@(wx)(�)@w(t)

]=−E[ Y (y(�))x(�− t)]; (9)

where Y (u) = (p′Y =pY )(u) is the so-called score function.

The derivative of the second term is

@@w(t)

{12�

∫ 2�

0log

∣∣∣∣∣+∞∑

t′=−∞w(t′)e−jt′�

∣∣∣∣∣ d�}=

12�

∫ 2�

0

1W (ej�)

e−jt� d�: (10)

Clearly, this second term is the {−t}th coe5cient of the inverse ;lter of w. In thefollowing, it will be noted Lw(−t).

The two last terms of (8) does not depends on w, then their contribution to thegradient of I(Y ) with respect to w is zero. Combining (9) and (10) leads to

@I(Y )@w(t)

=−E[x(�− t) Y (y(�))]− Lw(−t): (11)

J. Sole et al. / Neurocomputing 48 (2002) 339–355 343

From (11), we could deduce a simple gradient algorithm, but we prefer derive thenatural or relative gradient descent algorithms [1,5], which exhibit uniform perfor-mance. Denoting "y; Y (y) = E[y Y (y)] the high order cross-correlation between yand Y (y), the natural gradient, derived from (11), leads to the symbolic algorithm

w ← w + #{"y; Y (y) + �}w: (12)

More details on this algorithm are given in [15]. Moreover, an interactive JAVAsimulator is available on the author’s Web pages, whose address is provided at theend of the paper. Matlab routines can also be downloaded from this server.

4.2. Nonlinear subsystem

For this subsystem, we propose two diHerent parametric models for the functiong. The ;rst one is based on a polynomial model of g, and the second one on a1-hidden layer multilayer perceptron (MLP).

4.2.1. Polynomial parameterizationModeling g with a N -degree polynomial leads to

g(t) =N+1∑n=1

antn−1: (13)

From Fig. 1 and (13), we obtain

x(t) = g(e(t)) =N+1∑n=1

anen−1(t); (14)

y(t) = (wx)(t) =∞∑

i=−∞w(i)g(e(t − i)): (15)

The gradient descent algorithm for estimating g requires the derivatives of I(Y )(5) with respect to the coe5cients an of the polynomial. The derivatives of theright-side term of (5), with respect of the pth coe5cient, are

@H (y(�))@ap

=−E[

@@ap

logpY (y(�))]

=−E[ Y (y(�))

@y(�)@ap

]

=−E[ Y (y(�))

∞∑t=−∞

w(t)e(�− t)p−1

](16)

344 J. Sole et al. / Neurocomputing 48 (2002) 339–355

and

@H (Y )@ap

=@

@apE

[log

(N+1∑n=1

an(n− 1)e(�)n−2

)]

= E

(N+1∑

n=1

an(n− 1)e(�)n−2

)−1@g(e(�))

@ap

= E

(N+1∑

n=1

an(n− 1)e(�)n−2

)−1

(p− 1)e(�)p−2

; (17)

where Y (u) = (p′Y =pY )(u) is the so-called score function.

Combining (16) and (17) leads to

@I(Y )@ap

=−E[ Y (y(�))

+∞∑t=−∞

w(t)e(�− t)p−1

]

−E(N+1∑

n=1

an(n− 1)e(�)n−2

)−1

(p− 1)e(�)p−2

: (18)

Eq. (18) is the gradient of I(Y ) with respect to polynomial coe5cients ap, andwill be used for estimating g according to the gradient descent algorithm

a← a− #@I(Y )@a

: (19)

4.2.2. Neural network parameterizationIn this subsection, we model g using a multilayer perceptron with one hidden

layer

g[a; c; b; u] =N∑i=1

ai%(ciu− bi) (20)

and from Fig. 1 we obtain

x(t) = g(e(t)) =N∑i=1

ai%(cie(t)− bi); (21)

y(t) = (wx)(t) =∞∑

i=−∞w(i)g(e(t − i)): (22)

J. Sole et al. / Neurocomputing 48 (2002) 339–355 345

The gradient descent algorithm of g requires the derivatives of I(Y ) (5) withrespect to the network parameters a; c and b:(a) the derivatives of H (Y ) with respect to the various parameters are given below:

• with respect of ap

@H (Y )@ap

=@

@apE[log(g′(e(t)))]

= E

[1∑N

i=1 aici%′(cie(t)− bi)

@@ap

g′(e(t))

]

= E

[cp%′(cpe(t)− bp)∑Ni=1 aici%′(cie(t)− bi)

]; (23)

• with respect of cp

@H (Y )@cp

= E

[ap(e(t)cp%′′(cpe(t)− bp) + %′(cpe(t)− bp))∑N

i=1 aici%′(cie(t)− bi)

]; (24)

• with respect of bp

@H (Y )@bp

= E

[−apcp%′′(cpe(t)− bp)∑N

i=1 aici%′(cie(t)− bi)

]; (25)

(b) the derivatives of H (y(�)) with respect to the various parameters are givenbelow:

• with respect of ap

@H (y(�))@ap

=−E[

@@ap

log(pY (y(�)))]=−E

[ Y (y(�))

@y(�)@ap

]

=−E[ Y (y(�))

@@ap

∞∑t=−∞

w(t)

(N∑i=1

ai%(cie(�− t)− bi)

)]

=−E[ Y (y(�))

∞∑t=−∞

w(t)%(cpe(�− t)− bp)

]; (26)

• with respect of cp

@H (y(�))@cp

=−E[ Y (y(�))

∞∑t=−∞

w(t)ape(�− t)%′(cpe(�− t)− bp)

]; (27)

346 J. Sole et al. / Neurocomputing 48 (2002) 339–355

• with respect of bp

@H (y(�))@bp

=+E

[ Y (y(�))

∞∑t=−∞

w(t)ap%′(cpe(�− t)− bp)

]: (28)

By combining these derivatives, we obtain the gradient of the mutual informationI(Y ) (5) with respect to the network coe5cients

@I(Y )@ap

=−E[

cp%′(cpe(t)− bp)∑Ni=1 aici%′(cie(t)− bi)

]

−E

[ Y (y(�))

∞∑t=−∞

w(t)%(cpe(�− t)− bp)

]; (29)

@I(Y )@cp

=−E[ap(e(t)cp%′′(cpe(t)− bp) + %′(cpe(t)− bp))∑N

i=1 aici%′(cie(t)− bi)

]

−E

[ Y (y(�))

∞∑t=−∞

w(t)ape(�− t)%′(cpe(�− t)− bp)

]; (30)

@I(Y )@bp

= E

[apcp%′′(cpe(t)− bp)∑Ni=1 aici%′(cie(t)− bi)

]

+E

[ Y (y(�))

∞∑t=−∞

w(t)ap%′(cpe(�− t)− bp)

]: (31)

Eqs. (29–31) will be used for deriving gradient descent algorithms for estimatingthe MLP, which models g

a← a− #a@I(Y )@a

; c← c − #c@I(Y )@c

; b← b− #b@I(Y )@b

: (32)

5. Algorithm

5.1. Practical issues

For implementing e5cient algorithms, a few practical points must be addressed:

• order of ;lter w,• normalization,

J. Sole et al. / Neurocomputing 48 (2002) 339–355 347

• score function estimations,• cross-correlation estimations.

As in;nite dimension ;lters are not tractable, we constrain w as a FIR ;lter oforder Nw = 2N0 + 1. Without assumptions on the orders of causal and noncausalparts, we choose a symmetrical ;lter with N0 entries for the causal part, N0 entriesfor the strictly causal part. Then, at the beginning of the algorithm, the initializationof w is w(t) = �(t − N0 − 1).

The solutions based on independence cannot be unique. In fact, independence issatis;ed if (g ◦ f)(u) = (u; ∀( and if w(t)h(t) = )�(t − �); ∀); �. For cancellingindeterminacies, one must add two normalizations, consisting to scaling to one thesignal power, just after g(·) and after w. We also normalize the ;lter so that thecentral coe5cient is equal to one, according to w(t)← (w(t)=w(N0+1)). The delayindeterminacy is cancelled by initializing the ;lter as explained above.Many methods can be used for estimating the score functions (;rst step in the

main loop). Although direct estimation is possible [16], in this paper we usedindirect estimation, based on the well known kernel estimators [7] of the densityand of its derivative.Finally, the cross-correlation "y; Y (y) = Ey Y (y), used in (12), is estimated ac-

cording to an empirical mean.

5.2. Algorithm

Denoting E={e(1); e(2); : : : ; e(T )} the observation sequence, the blind inversionalgorithm based on the polynomial model (and similarly based on the MLP model)writes as:

Require E: e(t); t = 1; : : : ; TInitializationg(·) = 1w(t) = �(t − N0 − 1)

Main loopdo

estimation of the score function, Y (y)estimation of the gradient, (@I(Y )=@an), according to (18), for each nupdating of an, according to (19), for each noutput of the nonlinear subsystem: x(t) =

∑anen(t)

normalization of x(t)empirical estimation of cross-correlation "y; Y (y)

updating of the ;lter coe5cients according to (12)normalization of the ;ltercomputation of the current output y(t)normalization of y(t)

until convergence

348 J. Sole et al. / Neurocomputing 48 (2002) 339–355

5.3. Experiments

In order to prove the e5ciency of the previous algorithms, we consider an in-put i.i.d. random sequence s(t), ;ltered by a non-minimum phase FIR ;lter h =[ − 0:082; 0; −0:1793; 0; 0:6579; 0; −0:1793; 0; −0:082] and then distorted withf(u) = 0:1u + tanh(5u) (see Fig. 2). Note on Fig. 2 (right), the saturation dueto the function tanh(·) which strongly distorts the input signal. The frequency re-sponse of h (Fig. 3) emphasizes on the non-minimum phase nature of h.The algorithm was trained with a sample size of T = 1000. The length of the

impulse response of w is set to 21 with equal length for the causal and anti-causalparts. Estimation results, shown in Figs. 4 and 5, prove the e5ciency of the twoalgorithms (with polynomial or neural models) for estimating the inverse of nonlin-ear function f. In fact, Fig. 5 points out that estimated nonlinearity has the shapeof the inverse of tanh(·), and amplitude and phase responses cancel the channelcharacteristics shown in Fig. 4.The performance can be directly measured with the output signal noise ratio

S=N = E[y2(t)]=E(s(t)− y(t))2, where N is the error power and S the estimatedsignal power. After adequate processing (delaying and re-scaling of y(t)), oneobtains S=N ≈ +18 dB with both polynomial and neural models.

6. Discussion

In the previous experiments, the input signal was an i.i.d. random sequence.However, in real world situations, we usually handle non-i.i.d. signals. What willhappen with our inversion method if the input signal is no longer i.i.d.? In sucha situation, we will recover an i.i.d. version of the original, i.e. the innovationprocess of the original signal, if it exists. In fact, supposing that the signal can bemodeled as an i.i.d. sequence ;ltered with an AR ;lter, we can merge the AR ;lterand the channel ;lter in a single ;lter, as shown in Fig. 6. Then, the inversionsystem will recover the inverse of the cascade, and the output of the inversionsystem will be the i.i.d. sequence at the input of the AR ;lter, i.e. the so-calledinnovation process sr(t) instead s(t) (Fig. 7).This result is validated by another experiment (Fig. 8), using (as input signal

s(t)) a non-i.i.d. signal, obtained by ;ltering an i.i.d. random sequence, sr(t), withan AR ;lter. As expected, the inverse ;lter w estimated by the method is the inverseof the cascade (h(z)=AR(z)), and its output is the innovation process generating thedata, as showed in Fig. 8. In all ;gures, the spectrums are computed with FFT Mat-lab function with 1024 points, and only one of the two symmetric parts is showed.If we know the AR model of the signal s(t), we can easily recover this signal

using from the i.i.d. output of the inversion system, y(t) ≈ sr(t), by ;ltering y(t)with the AR ;lter (Fig. 7). Fig. 8c shows the recovered signal spectrum, after thispost-;ltering, becomes very close to the spectrum of s(t).In the next example we present a result obtained with a real music signal, which

is not an i.i.d. signal. We use the ;rst 1000 samples for estimating the inverse of

J. Sole et al. / Neurocomputing 48 (2002) 339–355 349

Fig. 2. From left to right, input signal s(t), ;ltered signal and distorted signal e(t).

350 J. Sole et al. / Neurocomputing 48 (2002) 339–355

Fig. 3. Frequency domain response and zeros of h.

Fig. 4. On the left, estimated inverse of h: w frequency domain response. On the right, estimatedinverse of f using a 10-deg polynomial model: x(t) vs. e(t).

Fig. 5. On the left, estimated inverse of h: w frequency domain response. On the right, estimatedinverse of f using a multiplayer perceptron with 6 neurons in the hidden layer: x(t) vs. e(t).

J. Sole et al. / Neurocomputing 48 (2002) 339–355 351

Fig. 6. Block diagram of the equivalent convolution system for non-i.i.d. signals.

Fig. 7. Block diagram of the inversion signal for non-i.i.d. signals. The true original signal s(t) isrecovered by ;ltering the i.i.d. output y(t) with the AR ;lter, which requires prior information on thesignal.

the Wiener system. In fact, due to the non-i.i.d. nature of the input signal, theoutput y(t) should be an estimate of the innovation process, sr(t), of the musics(t) (Fig. 9b and d). Then, by ;ltering y(t) with an AR ;lter (of course, it requiresprior information), one can obtain a good estimate of the original music signal,s(t), from this innovation process, as we can see in Fig. 9c. The main problemof the method is to get a good prior information on the AR model of the source.This point, including the robustness of the estimation of the AR ;lter obtainedwith similar signals instead the true one s(t), must be further investigated.

7. Conclusions

In this paper, two fully blind parametric procedures for the inversion of a nonlin-ear channel (Wiener system) were proposed. Contrary to other blind identi;cationprocedures, the system input is assumed to be non-Gaussian. In fact, the methodfails for Gaussian signals, because it requires higher (than 2) order statistics. Theinversion procedure, in both cases, is based on the minimization of the mutual in-formation rate of the inverse system output. For achieving optimal implementationof this criterion, whatever the signal distribution, the score function is estimated forproviding the best statistics. The estimation of g is done according to a parametricmodel, using either a polynomial model or a neural MLP model. Both models leadto good results, even in di5cult situations (hard nonlinearity and non-minimum

352 J. Sole et al. / Neurocomputing 48 (2002) 339–355

Fig. 8. Inversion of a non-i.i.d. signal. (a) Spectra of the original signal s(t) in solid line, and theobserved signal e(t) in dashed line. (b) Spectra of the original signal s(t) in solid line, and thedeconvolved signal y(t) in dashed line. One can observe that y(t) �= s(t). (c) Spectra of the originalsignal s(t) in solid line, and the post-processed deconvolved signal [1=AR(z)]y(t) = s(t) in dashedline. We can observe that the spectrum of s(t) is almost equal to the spectrum of s(t). (d) Spectraof the innovation process sr(t) of the original signal s(t) in solid line, computed with Matlab LPCfunction, and the deconvolved signal y(t) in dashed line. We can observe that the two spectrums arevery similar.

phase ;lter). If the input is non-i.i.d., but is a linear ;ltering of an i.i.d. noise(the so-called innovation), the output provides the innovation instead of the inputsignal s(t). The restitution of the input then requires the prior knowledge (or theestimation) of the AR ;lter generating the signal s(t).Web server. Java demo and Matlab codes are available at the following

addresses:http://www.uvic.es/projectes/SeparationSourceshttp://195.83.79.145/weblis1/demo/ICAdemo/index.html

J. Sole et al. / Neurocomputing 48 (2002) 339–355 353

Fig. 9. Inversion of a real music signal. (a) Spectra of the original signal s(t) in solid line, andthe observed signal e(t) in dashed line. (b) Spectra of the original signal s(t) in solid line, and thedeconvolved signal y(t) ≈ sr(t) in dashed line. (c) Spectra of the original signal s(t) in solid line,and the post-processed deconvolved signal [1=AR(z)]y(t) = s(t) in dashed line. We can observe thatthe spectrum of s(t) is almost equal to the spectrum of s(t). (d) Spectra of the innovation processof the original signal s(t) in solid line, computed with Matlab LPC function, and of the deconvolvedsignal y(t) in dashed line. We can observe that the two spectra are very similar.

Acknowledgements

This work has been in part supported by the DirecciPo General de Recerca dela Generalitat de Catalunya, by the European Union (BLInd Source Separationand applications: BLISS, European project IST-1999-14190), and by the Frenchproject: Statistiques avancPees et signal (SASI).

References

[1] S.I. Amari, Natural gradient works e5ciently in learning, Neural Comput. 10 (1998) 251–276.

354 J. Sole et al. / Neurocomputing 48 (2002) 339–355

[2] R. Bars, I. BQezi, B. Pilipar, B. Ojhelyi, Nonlinear and long range control of a distillation pilotplant, Identi;cation and System Parameter Estimation, 9th IFAC=IFORS Symposium Budapest,1990, pp. 848–853.

[3] S.A. Bellings, S.Y. Fakhouri, Identi;cation of a class of nonlinear systems using correlationanalysis, Proc. IEEE 66 (1978) 691–697.

[4] E.D. Boer, Cross-correlation function of a bandpass nonlinear network, Proc. IEEE 64 (1976)1443–1444.

[5] J.-F. Cardoso, B. Laheld, Equivariant adaptive source separation, IEEE Trans. Signal Process. 44(1996) 3017–3030.

[6] T.M. Cover, J.A. Thomas, Elements of Information Theory, Wiley Series in Telecommunications,1991.

[7] W. HSardle, Smoothing Techniques with implementation in S, Springer, Berlin, 1990.[8] I.W. Hunter, Frog muscle ;ber dynamic stiHness determined using nonlinear system identi;cation

technique, Biophys. J. (1985) 49–81.[9] I.W. Hunter, M.J. Korenberg, The identi;cation of nonlinear biological systems: Wiener and

Hammerstein cascade models, Biol. Cybernet. 55 (1985) 135–144.[10] G. Jacoviti, A. Neri, R. Cusani, Methods for estimating the autocorrelation function of complex

stationary process, IEEE Trans. ASSP 35 (1987) 1126–1138.[11] C.L. Nikias, A.P. Petropulu, Higher-Order Spectra Analysis—A Nonlinear Signal processing

Framework, Prentice-Hall, Englewood CliHs, NJ, 1993.[12] C.L. Nikias, M.R. Raghuveer, Bispectrum estimation: a digital signal processing framework, Proc.

IEEE 75 (1987) 869–890.[13] S. Prakriya, D. Hatzinakos, Blind identi;cation of LTI-ZMNL-LTI nonlinear channel models,

Biol. Cybernet. 55 (1985) 135–144.[14] A. Taleb, C. Jutten, Source separation in postnonlinear mixtures, IEEE Trans. Signal Process. 47

(1999) 2807–2820.[15] A. Taleb, J. SolPe, C. Jutten, Quasi-Nonparametric Blind Inversion of Wiener Systems, IEEE

Trans. Signal Process. 49 (2001) 917–924.[16] A. Taleb, C. Jutten, Entropy optimization. Application to blind source separation, in: W. Gerstner,

A. Germond, M. Hasler, J.-D. Nicoud (Eds.), Arti;cial Neural Networks-ICANN’97, LectureNotes in Computer Science, Vol. 1327, Springer, Berlin, 1997, pp. 528–534.

Jordi Sol,e received the telecommunications engineering diploma in 1995 andthe Ph.D. degree in 2000 from the Polytechnic University of Catalonia (UPC),Barcelona, Spain. In 1994, he joined the Department of Electronics and Tele-communications, University of Vic, Spain, where he is currently an AssociateProfessor. He was a Visiting Researcher with the Images and Signal Labo-ratory (LIS), Institut National Polytechnique, Grenoble, with which he has acontinuous close collaboration. His research interest are with statistical signalprocessing, neural networks, source separation, and music signal processing.

J. Sole et al. / Neurocomputing 48 (2002) 339–355 355

Christian Jutten received the Ph.D. degree in 1981 and the Docteur Qes Sci-ences degree in 1987 from the Institut National Polytechnique of Grenoble(INPG), France. He was an associate professor in Ecole Nationale SupPerieured’Electronique et de RadioPelectricitPe of Grenoble from 1982 to 1989. He wasvisiting professor in Swiss Federal Polytechnic Institute in Lausanne in 1989,before becoming full professor with the UniversitPe Joseph Fourier, Greno-ble, more precisely in the Sciences and Techniques Institute. He is associatedirector of the Images and Signal Laboratory (LIS). He has been associateeditor of IEEE Transactions on Circuits and Systems, and co-organizer withDr. J.-F. Cardoso and Prof. Ph. Loubaton of the 1st International Conferenceon Blind Signal Separation and Independent Component Analysis (Aussois,

France, January 1999). For 15 years, his research interest have been learning in neural networks, andblind source separation and independent component analysis.

Anisse Taleb received both the electrical engineering and the DEA degrees in 1996 from the InstitutNational Polytechnique of Grenoble (INPG), Grenoble, France, with specialization in signal, imageand speech processing. He received the Ph.D. degree from INPG in September 1999. From September1999 to January 2000, he was a Postdoctoral Research Fellow with the Images and Signal Laboratory(LIS), INPF, and a Research Fellow at the Australian Telecommunications Research Institute, Perth,Australia, from March 2000 to January 2001. He is now with Ericsson Research, Kista, Sweden. Hisresearch interest includes statistical signal processing, neural networks, and blind source separation.