8
* Correspondig author. Laboratoire associe H au C.N.R.S. (U.R.A. 1306), a` l'ENS, et aux Universite H s Paris VI et Paris VII. Tel.: #331-4432-3275; fax: #331-4432-3233. E-mail addresses: parga@ccuam3.sdi.uam.es (N. Parga), nadal@lps.ens.fr (J.-P. Nadal). Signal Processing 80 (2000) 2187}2194 Blind source separation with time-dependent mixtures Nestor Parga!, Jean-Pierre Nadal",* !Departamento de Fn & sica Teo & rica, Universidad Auto & noma de Madrid, Cantoblanco, 28049 Madrid, Spain "Laboratoire de Physique Statistique de l'ENS, Ecole Normale Supe & rieure, 24, rue Lhomond, 75231 Paris Cedex 05, France Received 24 June 1996; received in revised form 3 September 1998 Abstract We address the problem of blind source separation in the case of a time-dependent mixture matrix. For a slowly and smoothly varying mixture matrix, we propose a systematic expansion which leads to a practical algebraic solution when stationary and ergodic properties hold for the sources. ( 2000 Elsevier Science B.V. All rights reserved. Zusammenfassung Wir befassen uns mit dem Problem der blinden Quellentrennung bei einer zeitabha K ngigen Mischmatrix. Fu K r eine langsam und glatt vera K nderliche Mischmatrix schlagen wir eine systematische Entwicklung vor, die auf eine praktische algebraische Lo K sung bei stationa K ren and ergodischen Quelleneigenschaften fu K hrt. ( 2000 Elsevier Science B.V. All rights reserved. Re 2 sume 2 Nous conside H rons le proble`me de la se H paration aveugle de sources dans le cas d'une matrice de me H lange variant lentement et continuement avec le temps. Nous proposons un de H veloppement syste H matique conduisanta` une solution alge H brique dans le cas ou` les sources satisfont a` certaines conditions de stationarite H et d'ergodicite H . ( 2000 Elsevier Science B.V. All rights reserved. Keywords: Blind source separation 1. Introduction The problem of source separation arises in many di!erent "eld, both in signal processing (speech, radar, etc.) and in neural computation (`cocktail- partya e!ect, separation of odors, etc.). In all these cases one has to separate di!erent independent `sourcesa (voices, odors, etc.) that appear linearly superposed (mixed) when gathered by a set of sen- sors. Although the data have a linear structure, the di$culty of the task is that the `mixture matrixa, that is the set of coe$cients in the linear superposi- tion, is unknown } hence the name of `blind source separationa (BSS). Since the early proposals of Herault and Jutten [10] and of Bar-ness [2], a lot of e!ort have been devoted to the search of e$cient algorithms for performing BSS (see, e.g., 0165-1684/00/$ - see front matter ( 2000 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 5 - 1 6 8 4 ( 0 0 ) 0 0 0 7 6 - 1

Blind source separation with time-dependent mixtures

Embed Size (px)

Citation preview

Page 1: Blind source separation with time-dependent mixtures

*Correspondig author. Laboratoire associeH au C.N.R.S.(U.R.A. 1306), a l'ENS, et aux UniversiteH s Paris VI et Paris VII.Tel.: #331-4432-3275; fax: #331-4432-3233.

E-mail addresses: [email protected] (N. Parga),[email protected] (J.-P. Nadal).

Signal Processing 80 (2000) 2187}2194

Blind source separation with time-dependent mixtures

Nestor Parga!, Jean-Pierre Nadal",*!Departamento de Fn&sica Teo& rica, Universidad Auto& noma de Madrid, Cantoblanco, 28049 Madrid, Spain

"Laboratoire de Physique Statistique de l'ENS, Ecole Normale Supe& rieure, 24, rue Lhomond, 75231 Paris Cedex 05, France

Received 24 June 1996; received in revised form 3 September 1998

Abstract

We address the problem of blind source separation in the case of a time-dependent mixture matrix. For a slowly andsmoothly varying mixture matrix, we propose a systematic expansion which leads to a practical algebraic solution whenstationary and ergodic properties hold for the sources. ( 2000 Elsevier Science B.V. All rights reserved.

Zusammenfassung

Wir befassen uns mit dem Problem der blinden Quellentrennung bei einer zeitabhaK ngigen Mischmatrix. FuK r einelangsam und glatt veraK nderliche Mischmatrix schlagen wir eine systematische Entwicklung vor, die auf eine praktischealgebraische LoK sung bei stationaK ren and ergodischen Quelleneigenschaften fuK hrt. ( 2000 Elsevier Science B.V. Allrights reserved.

Re2 sume2

Nous consideH rons le probleme de la seH paration aveugle de sources dans le cas d'une matrice de meH lange variantlentement et continuement avec le temps. Nous proposons un deH veloppement systeHmatique conduisant a une solutionalgeH brique dans le cas ou les sources satisfont a certaines conditions de stationariteH et d'ergodiciteH . ( 2000 ElsevierScience B.V. All rights reserved.

Keywords: Blind source separation

1. Introduction

The problem of source separation arises in manydi!erent "eld, both in signal processing (speech,radar, etc.) and in neural computation (`cocktail-

partya e!ect, separation of odors, etc.). In all thesecases one has to separate di!erent independent`sourcesa (voices, odors, etc.) that appear linearlysuperposed (mixed) when gathered by a set of sen-sors. Although the data have a linear structure, thedi$culty of the task is that the `mixture matrixa,that is the set of coe$cients in the linear superposi-tion, is unknown } hence the name of `blind sourceseparationa (BSS). Since the early proposals ofHerault and Jutten [10] and of Bar-ness [2], alot of e!ort have been devoted to the search ofe$cient algorithms for performing BSS (see, e.g.,

0165-1684/00/$ - see front matter ( 2000 Elsevier Science B.V. All rights reserved.PII: S 0 1 6 5 - 1 6 8 4 ( 0 0 ) 0 0 0 7 6 - 1

Page 2: Blind source separation with time-dependent mixtures

Nomenclature

N number of inputs and sourcesS(t)"MS

j(t),

j"1,2, NNinput data

r(t)"Mpa(t),

a"1,2, NNUnknown sources

M"MMj,a

,a"1,2, Nj"1,2, NN

mixture matrix

J N]N matrix de"ning a linear "lterapplied to the data

da,b

Kronecker symbolS.T empirical expectation: average on

a given time window.S.T

csecond-order cumulant (e.g.SS

1S2Tc"SS

1S2T!SS

1TSS

2T

with S.T de"ned above)¹ length of time window on which

averaged are taken

q time delay used for measuringtime correlations

* typical time scale of the processgenerating the sources

K0"MK0a,b

,a"1,2, Nb"1,2, NN

second-order cross-cumulantmatrix of the sources

K(q)"MK(q)a,b

,a"1,2, Nb"1,2, NN

second-order cross-cumulantmatrix of the sources at a timedelay q

C0

second-order cross-cumulantmatrix of the input data

C(q) second-order cross-cumulantmatrix at a time delay q of theinput data

C`(q) symmetric part of the matrix C(q)X,O N]N orthogonal matricesK diagonal matrix (eigenvalues of C

0)

CA

second-order cumulant of anarbitrary quantity A.

[1,3,5,6,11}13]). In the standard case, the mixturematrix is a constant (it does not change with time).Then the linear structure of the data allows toperform BSS by applying some constant linear "l-ter to the output of the array of sensors, which iscomputed from some analysis of the statistics ofthese outputs.

In the present paper we address the issue of BSSfor time-dependent mixture matrices. This morecomplicated situation may arise if, e.g., the sourcesare moving with respect to the data collecting sys-tem [10]. Clearly, with no other prior knowledge,one cannot expect in that case to separate thesources at each instant of time. However, throughsome adaptive procedure, one may hope to obtaingood performances in average, or a reasonable pre-diction of the mixture matrix based on previousobservations. In this paper we propose such anapproach to blind source separation with a time-dependent mixture matrix for the particular situ-ation where the following properties are expectedto hold: (i) the time dependency of the mixturematrix is smooth and slow as compared to the

typical time scale of the sources; (ii) the sourcedynamics satisfy some stationary and ergodic prop-erties. More precisely, we will propose a systematicexpansion leading to an algebraic solution basedon the measure of a limited number of correlationsbetween a well chosen set of combinations of inputdata.

2. Blind source separation: A reminder

2.1. Linear mixtures of independent sources

The standard paradigm of BSS is the following.The input data are assumed to be a linear mixture ofindependent sources. More precisely, at each timet the observed data S(t) is an N-dimensional vectorgiven by

Sj(t)"

N+a/1

Mj,a

pa(t), j"1,2, N, (1)

2188 N. Parga, J.-P. Nadal / Signal Processing 80 (2000) 2187}2194

Page 3: Blind source separation with time-dependent mixtures

(in vector form S"Mr) where the pa

are N statist-ically independent variables, of unknown probabil-ity distributions, and M is an unknown N]Nmatrix, called the mixture matrix. In the simplestcase, M is time independent.

By hypothesis, all the source cumulants are diag-onal, in particular, the second-order cumulant atequal time K0 is of the form

K0a,b

,Spa(t)p

b(t)T

c"d

a,b,K0

a,a, (2)

where da,b

is the Kronecker symbol. Without loss ofgenerality, one can always assume that the sourceshave zero average:

SpaT"0, a"1,2,N (3)

(otherwise one has to estimate the average of eachinput, and substract it from that input).

Performing BSS means "nding the linear "lter,that is a N]N matrix J, such that the N-dimen-sional "lter ouput h

hi(t)"

N+j/1

Ji,j

Sj(t), i"1,2, N (4)

gives a reconstruction of the sources: ideally, onewould like to have J"M~1. However, as it is wellknown and clear from the above equations, one canrecover the sources only up to an arbitrary permu-tation, and up to a multiplicative factor of arbitrarysign for each source. In particular, this meansthat the cumulant K0 is arbitrary: one can al-ways assume the sources to have unit variance,K0

a,a"1, a"1,2,N.

As it is usually done in the study of source separ-ation, one assumes that the number of sources isknown (there are N observations, e.g. N captors, forN independent sources), and one assumes M to beinvertible. The di$culty comes from the fact thatthe statistics of the sources are not known, themixture matrix is not known and is not necessarily(and in general it is not) an orthogonal matrix.

2.2. BSS from time correlations

A lot of work has been done in order to de"nee$cient BSS algorithms (see, e.g., [1,3,5}8,10}13]).

We will not at all make a review of known algo-rithms. For our purpose, it will however be conve-nient to consider one particular technique, namelythe algebraic approach based on time correlations[4,9,11,13,14].

We thus assume that the second-order cross-cumulant matrix K(q) for some time delay q'0,

K(q)a,b

,Spa(t)p

b(t!q)T

c(5)

has non-zero diagonal elements:

K(q)a,b

"da,b

Ka,a

(q). (6)

Then J diagonalizes C0

and C(q) simultaneously,where C

0is the second-order cumulant at equal

times,

C0,SSSTT

c"MK0MT (7)

and C(q) is the second-order cumulant of the inputsat time delay q:

C(q),SS(t)ST(t!q)Tc"MK(q)MT. (8)

Finding J is then an easy to solve algebraic prob-lem [4,9,11,13,14]. One possible way for computingJ is to "rst perform the principal component analy-sis of the data (diagonalization of C

0), which deter-

mines J up to an orthogonal matrix [6]. Thisorthogonal matrix is then obtained as the onewhich diagonalizes the matrix C(q) projected ontothe principal components (see e.g. [13] for details).

3. Time-dependent mixtures

3.1. Formulation of the problem

Let us consider now the case of a time-dependentmixture matrix, M"M(t), which is a smoothly andslowly varying as compared to the typical timescale of the sources. More precisely, we assume thatthere exists some time scale ¹ such that, on anytime window of size ¹, one has a su$cient statisticsof the sources (that is their cross-cumulants esti-mated by averages over this time window are null),and the mixture matrix is almost constant: thenorm of the matrix dM/dt is small compared to

N. Parga, J.-P. Nadal / Signal Processing 80 (2000) 2187}2194 2189

Page 4: Blind source separation with time-dependent mixtures

1/¹. Here and in the following, average (at time t) ofa quantity A will thus mean an average over thetime window [t!¹, t], and we will compute it as

SA(t)TT,P

T

0

dt@¹

A(t!t@). (9)

Hence, we have for example

C0"SS(t)ST(t)T

c"SM(t)r(t)rT(t)MT(t)T

c

SS(t)ST(t)T"PT

0

dt@¹

M(t!t@)r(t!t@)

]rT(t!t@)MT(t!t@) (10)

and

C(q)"SS(t)ST(t!q)Tc

"SM(t)r(t)rT(t!q)MT(t!q)Tc,

SS(t)ST(t!q)T"PT

0

dt@¹

M(t!t@)r(t!t@)

]rT(t!q!t@)MT(t!q!t@).

(11)

3.2. Optimal constant xlter

If the time dependency of M is very weak, one cantry an adiabatic approximation: for each time win-dow [t!¹, t] one can compute a time-independent"lter matrix J

t. That is, on that particular time

window, one analyses the data as if they weregenerated by some linear mixture with a time-independent mixture matrix. The subscript t addedto J is a reminder that J, computed in this way, isassociated to this particular time interval [t!¹, t](since the true mixture matrix is evolving with time,the constant matrix J

t{that will be computed from

data of a di!erent time interval [t@!¹, t@] will bedi!erent). One may ask J

tto perform source separ-

ation in average over that time window [t!¹, t](e.g. one can ask for the outputs of the "lter to havecross-cumulants as small as possible when thesecumulants are computed from time averages overthis time window).

One possibility would be to compute the "ltermatrix as the one which minimizes some conve-nient criterium measuring the quality of source

separation. This is not what we will do, since hereand in the following we want to make an expliciteuse of the hypothesis that the mixture matrix isslowly varying. A convenient choice is then to com-pute J

tas the common set of left eigenvectors of the

two cumulants C0

and C(q), exactly as one woulddo it if M was time independent (see Section 2.2).But, in order to use the same technique as fora constant M, one has to deal with two symmetricmatrices. However, C(q) may not be a symmetricmatrix due to the time-dependency of M (and thisasymmetry is in fact a signature of the non-con-stancy of the mixture matrix). One can rather com-pute J

tfrom the diagonalization of C

0and C`(q),

where C`(q) is the symmetric part of C(q):

C`(q)"12[C(q)#C(q)T]. (12)

The strategy for computing Jtis then as follows (as

shortly explained in Section 2.2 and detailed in[13]). One "rst perform the principal componentanalysis, which means computing the orthogonalmatrix X such that C

0"XTKX where K is the

diagonal matrix whose diagonal elements are theeigenvalues of C

0. Then J

tis searched for as

Jt"OK~1@2X, where O is another orthogonal

matrix. This matrix is chosen as the one whichdiagonalizes the matrix C`(q) after projection ontothe principal components, that is the matrixK~1@2XC`(q)XTK~1@2. In such a way J

tC0

JTt

andJtC`(q)JT

tare diagonal matrices, which is the de-

sired result.In addition we will see that this way of comput-

ing a constant "lter matrix is precisely what weneed for the expansion we propose in the nextsection.

3.3. Towards a systematic expansion

In order to get a better estimate of the mixturematrix (or of its inverse) for a given time window[t!¹, t], one may try to estimate M(t!t@) for t@between 0 and ¹ in a linear expansion in t@. Insteadof computing one matrix J

t, one will then compute

two matrices. More generally, performing an ex-pansion up to some given order k in t@/¹, one willhave to compute k#1 matrices. In term of theassumptions presented in Section 3.1, as we will

2190 N. Parga, J.-P. Nadal / Signal Processing 80 (2000) 2187}2194

Page 5: Blind source separation with time-dependent mixtures

show in the next section such an expansion will leadto performing BSS on [t!¹, t] up to a given orderin ¹ dM/dt.

The above strategy requires to store the dataduring the time window [t!¹, t] in order to com-pute the optimal "lter matrix for that time window[t!¹, t]: this implies an ow line processing. Let ushowever comment brie#y on the use of such anapproach for online processing. One may use asa prediction for J

t`Tthe result of the computation

done on the previous window [t!¹, t]. Then it isclear that, for instance, the optimal constant "ltermatrix J

t`Twill be obtained from the linear expan-

sion performed on [t!¹, t]. More generally, onemay compute J for the time window [t, t#¹] atorder k in t@/¹ from an expansion at order k#1 on[t!¹, t].

We now come back to the (o!-line) processing ofdata for one given window, dealing with the linearexpansion.

4. First-order expansion

4.1. Linear approximation within a time window

The hypothesis on the slow evolution of themixture matrix means that, at "rst order, M(t!t@)for t@ within 0 and ¹ can be written as

M(t!t@)"M0t!

t@¹

M1t, (13)

where M0t

and M1t"¹dM/dt are two unknown

matrices to be determined from statistics on thetime window [t!¹, t]. What we want to do is tosee whether one can measure a limited number ofcorrelations of the input data in order to determineM0

tand M1

t. Let "rst consider the second-order

cumulant matrix at some time delay q, C(q) asde"ned in (11) (note that C(q"0),C

0). Replacing

M given by (13), taking q smaller than ¹, with theaverages de"ned as in (9), we have

SS(t)ST(t!q)T"PT

0

dt@¹

MM0tr(t!t@)rT(t!q!t@)M0T

t

!

t@¹

M1tr(t!t@)rT(t!q!t@)M0T

t

!

t@¹

M0tr(t!t@)rT(t!q!t@)M1T

t

!

M0tr(t!t@)rT(t!q!t@)M1T

tN

(14)

and one gets

C(q)"M0tK(0)(q)M0T

t!M1

tK(1)(q)M0T

t

!M0tK(1)(q)M1T

t!

M0tK(0)(q)M1T

t, (15)

where the K(k)(q) are the generalized source cumu-lants:

K(k)(q),PT

0

dt@¹ A

t@¹B

kr(t!t@)rT(t!q!t@)

!CPT

0

dt@¹

r(t!t@)D]CP

T

0

dt@¹ A

t@¹B

krT(t!q!t@)D. (16)

Note that K(0)(q),K(q). One can see from the lastterm in the r.h.s of (15) that C(q) is indeed notsymmetric for non-zero q. From the above equa-tions, it appears that measuring C

0and C(q) will

not be su$cient in order to estimate M0t

and M1t,

since we have K(1)(q) as an additional unknown.However, measuring, say, S(t@/¹)S(t!t@)ST(t!q!t@)T

c, will not help, since this average will de-

pend on a new source cumulant, namely K(2)(q).

4.2. A tractable case

We now make use of the assumption that thesources are generated according to some stationaryand ergodic process (at least up to the second-orderstatistics), together with the hypothesis that thetime window ¹ is large compared to the typicaltime scale of this process.

Let us consider some quantity A(t) of interest,such as the vector r(t) or the matrix r(t)rT(t!q),

N. Parga, J.-P. Nadal / Signal Processing 80 (2000) 2187}2194 2191

Page 6: Blind source separation with time-dependent mixtures

and the integral

Z(t)"PT

0

dt@¹

fAt@¹BA(t!t@), (17)

where f (u) is any positive function de"ned on [0,1],such as f (u)"uk for k integer. A being a randomvariable, we can consider its average value and the#uctuations around it. From ergodicity we have

SZT"SATP1

0

du f (u), (18)

where the average SAT does not depend on the timet (stationarity).

Now, let us consider the #uctuations around themean as characterized by the second cumulant:

SZ2Tc,SZ2T!SZT2

"PT

0

dt1f (t

1)P

T

0

dt2

f (t2)

SA(t!t1)A(t!t

2)T

c. (19)

From the stationarity hypotheses the second-ordercumulant matrix C

A,SA(t!t

1)A(t!t

2)T

cis

a function of the time di!erence alone:

SA(t!t1)A(t!t

2)T

c"C

A[v,t

1!t

2]. (20)

One then has

SZ2Tc"P

1

0

du f (u)P1

0

du@ f (u@)CA[¹(u!u@)]. (21)

In order to characterize the typical time scale of theprocess, we make the more explicit hypotheses thatfor some a and D, positive and "nite numbers, thenorm of the correlation matrix is bounded accord-ing to

for any v DCA[v]D(aWC

DvDD D, (22)

where W(x)"W(!x) is some function which goesquickly to zero as DxD goes to in"nity, e.g.,

W(x)"exp!DxD2r (23)

for some r'0. We may also assume that D is takenas the smallest value for which (22) holds. Hence wehave

SZ2Tc(aP

1

0

du f (u)P1

0

du@ f (u@)WA¹

DDu!u@DB. (24)

Our hypothesis for ¹ means that ¹ is largecompared to D;

¹AD. (25)

At lowest order, that is in the limit¹/DPR, SZ2T

cvanishes, so that Z is a non-

#uctuating quantity: it is almost surely equal to itsmean value SZT. In replacing Z by SZT, one is in

fact neglecting terms (at worst) of order JD/¹.More precisely, for ¹/D large one can write

WA¹

DDvDB"

ad(v)#CD¹D

3bdA(v) (26)

with d the Dirac distribution and

a"P=

~=

dxW(x), b"P=

~=

dx W(x)x2. (27)

As a result,

DSZ2TcD&

a aP1

0

du f (u)2. (28)

In the above derivation we have been workingwith A as a function of t alone. In the case ofa quantity such as A"r(t)rT(t!q), A is in facta function of t and t!q. One can make exactly thesame analysis as above, in which one will get ex-pressions with, e.g., W((¹/D)Du!u@!q/¹D). Hence,the derivation will apply as well if the limit of large¹/D is taken at a given value of q/¹. This is equiva-lent to state that the scaling regime of interest is

D@q@¹ (29)

so that one can neglect terms of order D/¹ evenwhen taking into account terms of order q/¹.

4.3. Solution at xrst order

According to the above discussion one can re-place, under the integrals de"ning the cumulants

2192 N. Parga, J.-P. Nadal / Signal Processing 80 (2000) 2187}2194

Page 7: Blind source separation with time-dependent mixtures

K(k)(q), the source terms by their average, that is

K(k)(q)"PT

0

dt@¹ A

t@¹B

kK(0)(q)"

1

k#1K(0)(q). (30)

As a result, correlations at a time delay q involveonly K(0)(q). Moreover, one sees that the only prod-ucts of matrices that appear for a given q areM0

tK(0)(q)M0T

t, M0

tK(0)(q)M1T

tand M1

tK(0)(q)M0T

t.

Since K0,K(0)(0) is arbitrary as explained in Sec-tion 2.1, we need to measure only three combina-tions of correlations. Denoting by C(1)(q) thecumulants

C(1)(q)"PT

0

dt@¹

t@¹

S(t!t@)ST(t!q!t@)

!CPT

0

dt@¹

S(t!t@)D]CP

T

0

dt@¹

t@¹

ST(t!q!t@)D (31)

and adding a subscript # (resp. !) to denote thesymmetric (resp. antisymmetric) part of a matrix,one obtains easily the following relations:

4C0!6C(1)

0"M0

tK0M0T

t, (32)

A4#3q¹BC`(q)!6A1#

q¹BC(1)`(q)

"M0tK(0)(q)M0T

t(33)

and

C~(q)#3q¹

MC`(q)!2C(1)`(q)N

"

M1tK(0)(q)M0T

t. (34)

The matrices M0t

and K(0)(q) are obtained from the"rst two Eqs. (32) and (33), where the data appearon the r.h.s. in the form of two symmetric matrices:to get M0

tand K(0)(q) one can then use exactly the

same techniques as those used in order to obtainthe matrices M and K(q) for a time-independentmixture, see Section 2.2. Eventually, M1

tis easily

computed from Eq. (34).

Two "nal remarks.f One should note that the l.h.s. of Eqs. (32)}(34)

are cross-correlations of particular combinationsof the input data: one can then measure thesecorrelations directly, instead of computing separ-ately each matrix appearing in these equations.

f As a de"nition of the average S.T, one may useinstead of (9), any similar de"nition such as

SA(t@)Tt,P

=

0

dt@W(t@)A(t!t@), (35)

where W(t@) is su$ciently zero for t@ larger than ¹.The above derivation can be easily adapted, pro-vided W is such that the numerical factors:=0

dt@W(t@)(t@/¹)k do depend on k } if not, allthe correlations C(q),C(1)(q) give the sameequation, and thus one cannot combine themto get formulae similar to (32)}(34). Note thatthis excludes in particular the choiceSA(t@)T

t,:=

0dt@/¹ exp(!t@/¹)A(t!t@).

5. Conclusion

In this paper we considered the problem of BSSwith a time-varying mixture. We showed that, un-der some assumptions on the statistics of the sour-ces, when the mixture matrix is slowly varying it ispossible to obtain a solution making use of tech-niques derived for time-independent mixtures. Weproposed a systematic expansion, and we presentedin detail the evaluation of the "rst-order expansion.The validity of the expansion can be checked a pos-teriori: one has to check that the "rst-order termM1

t, which is, in fact, ¹ dM/dt, is indeed small, and

whether the outputs do appear independent on anytime window of size ¹. As we explained it, theexpansion at a given order k, performed on a giventime window [t!¹, t], can be used either for owline processing } that is in order to process the dataof that time window } or for on line processing,predicting the mixture matrix at order k!1 for thenext time window.

It is cumbersome, but not di$cult, to derive theequations at any order } one has just to follow thesame strategy, expanding the mixture matrix up to

N. Parga, J.-P. Nadal / Signal Processing 80 (2000) 2187}2194 2193

Page 8: Blind source separation with time-dependent mixtures

the required order within a time window, as ex-plained in Section 3.3.

For simplicity, we have worked with second-order cumulants at equal time and at some delayq (q(¹). It would be interesting to adapt ourmethod to other approaches to BSS. For example,one may use criteria based on higher cumulants atequal time which have been proposed for perform-ing BSS in the case of a constant mixture matrix[6,7,13]. With averages de"ned as time-windowaverages as above (Section 3.1), one has to performthe necessary expansions of the cumulants underconsideration. Again, the zeroth order will begiven by the solution of the system as if M wasconstant.

Numerical simulations remain to be done in or-der to test the e$ciency of the proposed expansion.

Acknowledgements

This work has been partly supported by theFrench-Spanish program `Picassoa, the E.U. grantCHRX-CT92-0063, the Universidad AutoH nomade Madrid, the UniversiteH Paris VI and EcoleNormale SupeH rieure. NP and JPN thank, respec-tively, the Laboratoire de Physique Statistique(ENS) and the Departamento de FmH sica TeoH rica(UAM) for their hospitality.

References

[1] S.I. Amari, A. Cichocki, H.H. Yang, A new learning algo-rithm for blind signal separation, in: D.S. Touretzky, M.C.Mozer, M.E. Hasselmo (Eds.), Advances in Neural

Information Processing Systems, Vol. 8, MIT Press,Cambridge, MA, 1996.

[2] Y. Bar-Ness, Bootstrapping adaptive interference can-celers: some practical limitations, Proceedings of theGlobecom Conference Miami, November 1982, PaperF3.7, pp. 1251}1255.

[3] A. Bell, T. Sejnowski, An information-maximisation ap-proach to blind separation and blind deconvolution, Neu-ral Comput. 7 (1995) 1129}1159.

[4] A. Belouchrani, K. Abed Meraim, SeH paration aveugle ausecond ordre de sources temporellement correH leH es, Pro-ceedings of the GRETSI'93, Juan-Les-Pins, 1 (1993)309}312.

[5] G. Burel, Blind separation of sources: a nonlinear neuralalgorithm, Neural Networks 5 (1992) 937}947.

[6] J.-F. Cardoso, Source separation using higher-order mo-ments, Proceedings of the International Conference onAcoustics, Speech and Signal Processing-89, Glasgow,1989, pp. 2109}2112.

[7] P. Comon, Independent component analysis, a new con-cept?, Signal Processing, 36 1994 (287}314).

[8] N. Delfosse, Ph. Loubaton, Adaptive blind separation ofindependent sources: a de#ation approach, Signal Process-ing 45 (1995) 59}83.

[9] L. FeH ty, MeH thodes de traitement d'antenne adapteH e auxradio-communications, Ph.D. Thesis, ENST Paris, 1988.

[10] C. Jutten, J. Herault, Blind separation of sources, part i: anadaptive algorithm based on neuromimetic architecture,Signal Processing 24 (1991) 1}10.

[11] L. Molgedey, H.G. Schuster, Separation of a mixture ofindependent signals using time delayed correlations, Phys.Rev. Lett. 72 (1994) 3634}3637.

[12] J.-P. Nadal, N. Parga, Nonlinear neurons in the low noiselimit: a factorial code maximizes information transfer, Net-work 4 (1994) 565}582.

[13] J.-P. Nadal, N. Parga, Redundancy reduction and inde-pendent component analysis: conditions on cumulants andadaptive approaches, Neural Comput. 9 (7) (1997)1421}1456.

[14] L. Tong , V. Soo, R. Liu, T. Huang, Amuse: a new blindidenti"cation algorithm, Proceedings of the ISCAS, NewOrleans, USA, 1990.

2194 N. Parga, J.-P. Nadal / Signal Processing 80 (2000) 2187}2194