Embed Size (px)
Citation preview
Pattern Recognition Letters 33 (2012) 559–567
Contents lists available at SciVerse ScienceDirect
Pattern Recognition Letters
journal homepage: www.elsevier .com/locate /patrec
Resolving permutation ambiguity in correlation-based blind image separation
Kenji Hara ⇑, Kohei Inoue, Kiichi UrahamaDepartment of Visual Communication Design, Kyushu University, 4-9-1 Shiobaru, Minami-ku, Fukuoka-shi 815–8540, Japan
a r t i c l e i n f o a b s t r a c t
Article history:Received 27 December 2010Available online 8 December 2011Communicated by G. Moser
Keywords:Separation of reflectionBlind image separationPermutation ambiguityGeneralized multiple correlationPruning scheme
0167-8655/$ - see front matter � 2011 Elsevier B.V. Adoi:10.1016/j.patrec.2011.11.011
⇑ Corresponding author.E-mail address: [email protected] (K. Ha
We address the problem of permutation ambiguity in blind separation of multiple mixtures of multipleimages (resulting, for instance, from multiple reflections through a thick grass plate or through two over-lapping glass plates) with unknown mixing coefficients. In this paper, first we devise a generalized multi-ple correlation measure between one gray image and a set of multiple gray images and derive adecorrelation-based blind image separation algorithm. However, many blind image separation methods,including this algorithm, suffer from a permutation ambiguity problem that the success of the separationdepends upon the selection of permutations corresponding to the orders of the update operations. Tosolve the problem, we improve the first algorithm above by decorrelating the mixtures while searchingfor the appropriate update permutation using a pruning technique. We show its effectiveness throughexperiments with artificially mixed images and real images.
� 2011 Elsevier B.V. All rights reserved.
1. Introduction
When a scene is photographed through a glass plate, a reflectionis commonly observed superimposed over the scene. Especially inthe case of through a thick grass plate or through two overlappingparallel glass plates, multiple reflections are projected upon thescene. The separation of each reflection and transparency imagesfrom the photo(s) has recently received considerable attention inthe image processing and computer vision communities.
The existing image separation techniques can be broadly classi-fied into two categories: general blind source separation (Cardosoand Souloumiac, 1993; Bell and Sejnowski, 1995; Hyvaerinen andOja, 1997; Farid and Adelson, 1999; Cardoso et al., 2002; Bronsteinet al., 2005; Bedini et al., 2005; Kopriva, 2007; Gribonval andZibulevsky, 2010) and image-specific separation (Nayar et al., 1996;Lin and Shum, 2001; Levin et al., 2004; Sarel and Irani, 2004;Diamantaras and Papadimitriou, 2005; Tonazzini et al., 2006; Levinand Weiss, 2007; Kayabol et al., 2009; Hara et al., 2009). The former ismainly based on independent component analysis (ICA) (Hyvaerinenand Oja, 1997; Cichocki and Amari, 2002) and is applied morebroadly to other types of signals such as acoustics and biomedicalsignals. The latter does not necessarily use the ICA framework andis often restricted to separation of image mixtures. As examples ofstudies in the latter category, Levin et al. proposed automatic (Levinet al., 2004) and semi-automatic (Levin and Weiss, 2007) methodsfor separating layers from only a single mixed image based on a
ll rights reserved.
ra).
sparsity prior on natural images. Diamantaras et al. (Diamantarasand Papadimitriou, 2005) presented a robust and non-iterativemethod for separating image mixtures through the properties ofthe mixtures ratio. Sarel et al., in (Sarel and Irani, 2004), successfullyseparated linear mixtures of images by a decorrelation-based infor-mation exchange between mixtures. Tonazzini et al. (2006) accom-plished the blind separation of images from noisy linear mixtures byformulating as a Bayesian estimation problem.
Especially for the case of mixtures of more than three images,most of existing blind image separation methods suffer from a per-mutation ambiguity problem that the success of the separationdepends upon the selection of permutations corresponding to theorders of the update operations. In this paper, we present a methodfor resolving this permutation ambiguity in blind image separation.
First, Sarel et al.’s method (Sarel and Irani, 2004) will beextended to the blind separation of mixtures of an arbitrary numberof images. We devise a generalized multiple correlation measurebetween one gray image and a set of multiple gray images. Thismultiple correlation leads us to provide a set of simultaneous lin-ear equations for updating each mixture of images. Then sourceimages are estimated by iterating between solving the sets ofequations and cyclically permuting the mixtures of images.
However, there does also exist the permutation ambiguity, justas that in the other separation methods. Thus, we further improvethe first algorithm above by decorrelating the mixtures whilesearching for the appropriate update permutation using a pruningtechnique based on the discrepancy among the separation param-eter values for each of the three RGB channels. We show its effec-tiveness through experiments with artificially mixed images andreal images.
560 K. Hara et al. / Pattern Recognition Letters 33 (2012) 559–567
2. Decorrelation-based blind separation of image mixtures
In this section, we describe an extension of the two-layer sepa-ration method in (Sarel and Irani, 2004), adapted to the multi-layerseparation. This algorithm is included in the category of image-specific separation, and thus we describe it in the specific frame-work of color image analysis.
2.1. Problem formulation
Let ak = [ak,1 . . . ,ak,K]T (k = 1, . . . ,K) be the kth K-dimensional coef-ficient vector. Let IR
k ; IGk and IB
k (respectively LRk ; LG
k and LBk) be the
three RGB component images (the image size is M = h � w pixels)of the kth color mixed image Ik (respectively the kth color source
image Lk). Let IVk ¼ IR
k IGk IB
k
h i(respectively LV
k ¼ LRk LG
k LBk
h i) (k = 1, . . . ,K)
be the grayscale image (the image size N = 3M = h � 3w pixels)formed by horizontally concatenating IR
k ; IGk and IB
k (respectively
LRk ; LG
k and LBk). Then any kth grayscale mixed image IV
k is given bylinearly combining K independent grayscale source imagesLV
1 ; . . . ; LVK as,
IV1 ðiÞ ¼ a1;1LV
1 ðiÞ þ a1;2LV2 ðiÞ þ � � � þ a1;K LV
K ðiÞ;IV2 ðiÞ ¼ a2;1LV
1 ðiÞ þ a2;2LV2 ðiÞ þ � � � þ a2;K LV
K ðiÞ;...
IVK ðiÞ ¼ aK;1LV
1 ðiÞ þ aK;2LV2 ðiÞ þ � � � þ aK;K LV
KðiÞ;ði ¼ 1; . . . ;NÞ;
ð1Þ
where IVk ðiÞ (respectively LV
k ðiÞ) are the values of the ith pixels in IVk
(respectively LVk ), for k = 1, . . . ,K. For now, for given I1, . . . , IK with un-
known a1, . . . ,aK, we will estimate (constant times each of) the mostlikely L1, . . . ,LK.
2.2. Generalized multiple correlation
By extension of the generalized normalized grayscale correla-tion (GNGC) measure (Sarel and Irani, 2004), which is a correlationmeasure between two grayscale images of the same window size,we introduce a multiple correlation measure, which we will referto as the multiple generalized normalized grayscale correlation(MGNGC) measure, between one grayscale image fK and a set ofK � 1 grayscale images f1, . . . , fK�1 of the same window size N, as
MGNGCKðf1; . . . ; fK�1; fKÞ ¼PN
i¼1 MNGCiKðf1; . . . ; fK�1; fKÞ
h i2QKk¼1ViðfkÞPN
i¼1
QKk¼1ViðfkÞ
;
ð2Þ
where MNGCiKðf1; . . . ; fK�1; f KÞ is the multiple correlation coefficient
between the 5 � 5 sliding window centered at a pixel i in fK and theset of 5 � 5 sliding windows at the same pixel i in f1, . . . , fK�1. It iswell known that MNGCK,i(f1, . . . , fK�1; fK) is expressed as
MNGCiKðf1; . . . ; fK�1; fKÞ ¼ 1� 1eri
KK
� �1=2
; ð3Þ
where ~riKK is the KKth entry of the inverse matrix of the K dimen-
sional square correlation matrix RiK whose klth entry is given by
RiK
� �kl¼ Ciðfk; glÞ½ViðfkÞViðglÞ�
1=2 ðk; l ¼ 1; . . . ;KÞ; ð4Þ
where Ci(fk,gl) is the covariance between the ith windows in fk andgl, and Vi(fk) (respectively Vi(gl)) is the variance of the ith window infk (respectively gl).
2.3. Decorrelating multiple mixtures using cyclic iteration
In this section, we derive the update equation for estimating theimage sources from a given set of image mixtures. In each iterativeupdate we select a reference image cyclically from the image mix-tures and then subtruct the other image mixtures from the refer-ence image by weighting them with the related correlationmeasure MGNGC introduced in the previous section. Note thatthe weighting parameters need to be determined optimally suchthat the updated reference image and the image sources are decor-related. The exact details of how this determination of the weight-ing parameters is done are described below.
We will find a minimizer, rV ¼ rV1 � � �rV
K�1
� �T , of the MGNGCmeasure between the reference image eIV ¼ IV
K �PK�1
k¼1 rkIVk and
a set of K � 1 the image mixtures IV1 ; . . . ; IV
K�1, as follows (seeAPPENDIX for derivation).
rV1 � � �rV
K�1
� �¼ argminr1 ;...;rK�1
MGNGCK IV1 ; . . . ; IV
K�1;eIV� �
¼ argminr1 ;...;rK�1
�PN
i¼1 uiK�1
� T U iK�1
� ��1ui
K�1
QK�1k¼1 Vi IV
k
� �PN
i¼1ViðeIV ÞQK�1
k¼1 Vi IVk
� � ; ð5Þ
where U iK�1 is the K � 1 dimensional square correlation matrix
whose klth entry is U iK�1
� �kl¼ Ci IV
k ;IVlð Þ
Vi IVkð ÞVi IV
lð Þ½ �1=2 and uiK�1 is the K � 1
dimensional vector whose kth entry is uiK�1
� k ¼
Ci IVk ;eIV
� Vi IV
kð Þ½ �1=2
Hence, setting the partial derivatives of Eq. (5) with respect torV
k , for k = 1, . . . ,K � 1, to zero, we get (see APPENDIX forderivation)
XK�1
m¼1
XN
i¼1
Ci IVk ; I
Vm
� �YK�1
l¼1
Vi IVl
� �" #rV
m
¼XN
i¼1
Ci IVk ; I
VK
� �YK�1
l¼1
Vi IVl
� �ðk ¼ 1; . . . ;K � 1Þ: ð6Þ
Eq. (6) is equivalent to a set of K � 1 linear simultaneous equationswith unknowns rV ¼ rV
1 � � �rVK�1
� �T . The update equations for rV isobtained analytically as,
rV ¼ ðAV Þ�1bV; ð7Þ
where AV is the K � 1 dimensional square matrix and bV is the K � 1dimensional vector, as
ðAV Þkm ¼XN
i¼1
Ci IVk ; I
Vm
� �YK�1
l¼1
Vi IVl
� �; ðk;m ¼ 1; . . . ;K � 1Þ; ð8Þ
ðbV Þk ¼XN
i¼1
Ci IVk ; I
VK
� �YK�1
l¼1
Vi IVl
� �; ðk ¼ 1; . . . ;K � 1Þ: ð9Þ
Given as input a set of K mixed images I1, . . . , IK, our presentedalgorithm uses a two-step iterative procedure where the K � 1dimensional vector rV
k is updated, and subsequently the K imagesequence is cyclicly updated (the group of K successive iterationsis hereinafter referred to as a cycle), where the convergence crite-rion is such that all values of rV
1 � � �rVK�1 are less than 10�3, as out-
lined in Algorithm 1 and Fig. 3.
Table 1Interference-to-signal ratio (ISR) of image separation.
Condition number of mixingmatrix
JADE RelativeNewton
Algorithm 1(proposed)
5.37 (well-conditioned) 0.77 0.19 0.1346.77 0.77 0.19 0.23124.80 (ill-conditioned) 0.77 0.19 0.59
K. Hara et al. / Pattern Recognition Letters 33 (2012) 559–567 561
Algorithm 1.
Initialize IV1 ; . . . ; IV
K
repeatrV � (AV)�1bV
where
ðAV Þkm ¼XN
i¼1
Ci IVk ; I
Vm
� �YK�1
l¼1
Vi IVl
� �ðbV Þk ¼
XN
i¼1
Ci IVk ; I
VK
� �YK�1
l¼1
Vi IVl
� �ðk;m ¼ 1; . . . ;K � 1Þ
IV1
IV2
..
.
IVK
2666664
3777775(IVK �
PK�1
k¼1ðrV ÞkIV
k
IV1
..
.
IVK�1
266666664
377777775until convergencereturn I1, . . . , IK
For comparison, Table 1 shows the numerical error measures,interference-to-signal ratio (ISR), achieved by Algorithm 1 andtwo of the most efficient blind source separation methods – JADE(Cardoso and Souloumiac, 1993) and Relative Newton method(Gribonval and Zibulevsky, 2010), under different condition numbers(the ratio between the maximum and minimum eigenvalues) ofthe mixing matrices. As we can see in Table 1, while JADE andRelative Newton method do not depend on the condition numberof mixing matrix,1 Algorithm 1 deteriorates in the ill-conditioned case(condition number = 46.77,124.80). However, in the well-conditionedcase (condition number = 5.37), Algorithm 1 seems to outperformthe two other methods. Fig. 1 shows the separation results corre-sponding to the top row (well-conditioned case) of Table 1.
In each of JADE, Relative Newton, and our Algorithm 1, K! per-mutations are permitted for the K mixtures. For example, in Algo-rithm 1, it is possible for the resulting images to be negative undersome permutations. Algorithm 1 often leads to improper solutionsincluding these negative images. In our preliminary experiment wediscovered and confirmed that JADE and Relative Newton also suc-ceed or fail, depending upon the selection of permutations corre-sponding to the orders of the update operations. In each ofFig. 1(i)–(l), (m)–(p) and (q)–(t), shown previously, the visuallydetermined optimum permutation is used. The failed examplesare shown in Fig. 2((a), (c), (d), (g), (h) and (i) seem to be failures).It is shown in the following section that the problem in ourAlgorithm 1 is easily solved with a pruning scheme.
3. Simultaneous determination of update permutation
In this section we modify Algorithm 1 to decorrelate themixtures while searching for the appropriate update permutation.
Now, with the subscripts V and N(=h � 3w) in Algorithm 1replaced by C and M(=h � w), where the subscript C indicates eachof the three color channels (R,G,B), let rV obtained by executing the
1 It is known that at low noise levels the performance of invariant algorithms suchas JADE and Relative Newton method does not depend upon the mixing matrix(Cardoso, 1994).
first step (Eq. (7)) in Algorithm 1 for each color channel image se-quence IC
1 ; . . . ; ICK be rC ¼ rC
1 � � �rCK�1
� �TðC ¼ R;G; BÞ. With the help ofexperiments, we confirmed the fact that the equality relationamong rR, rG and rB depends upon the update permutation (notethat rR � rG � rB is not always satisfied for all the update permu-tations) and that Algorithm 1 tends to fail when the permutation ischosen such that the variation among rR, rG and rB is large. Thisobservation leads us to a pruning-based blind separation algo-rithm, stated in the following.
At first, for each of all the possible K! update permutations, therevised algorithm, as presented in Algorithm 2, individually re-peats the two-step process in Algorithm 1 for J iterations (or Q cy-cles with Q = J/K) and then, for the pth update permutation, recordsthe average value
TðpÞ ¼ 1J
XJ
j¼1
TjðpÞ ðp ¼ 1; . . . ;K!Þ; ð10Þ
where Tj(p) is, for the jth repetition of the two-step process underthe pth permutation,
TjðpÞ ¼krRj ðpÞ � rG
j ðpÞk þ krGj ðpÞ � rB
j ðpÞk þ krBj ðpÞ � rR
j ðpÞkðj ¼ 1; . . . ; JÞ; ð11Þ
where k � k denotes K � 1 dimensional Euclidean distance (L2 norm)and rC
j ðpÞðC ¼ R;G;BÞ is the value of rC ¼ rC1 � � �rC
K�1
� �T obtained forthe jth repetition of the two-step process under the pth permuta-tion. If TðpÞ exceeds a certain threshold value, the correspondingupdate permutation p is rejected from the set of the candidateupdate permutations. The above process is iterated until only onepermutation candidate p⁄ remains. On termination, the algorithmgives the estimated source images Ip� ;1 ¼ IR
p� ;1IGp� ;1IB
p� ;1
h i; . . . ; Ip� ;K ¼
IRp� ;K IG
p� ;K IBp� ;K
h i.
Algorithm 2
P ( fð1;2; . . . ;KÞ|fflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflffl}1
; ð2;1; . . . ;KÞ|fflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflffl}2
; . . . ; ðK; . . . ;2;1Þ|fflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflffl}K!
g
for all p such that p 2 P doInitialize IC
1ðpÞ; . . . ; ICKðpÞ;C ¼ R;G;B, for p
end for
repeatfor all p such that p 2 P dofor j = 1 to J do
for all C such that C 2 {R,G,B} do
rCj ðpÞ ( ðA
CðpÞÞ�1bCðpÞ
IC1ðpÞ
IC2ðpÞ
..
.
ICKðpÞ
266664377775(
ICKðpÞ �
PK�1
k¼1ðrC
j ðpÞÞkICk ðpÞ
IC1ðpÞ
..
.
ICK�1ðpÞ
26666664
37777775end for
Fise
562 K. Hara et al. / Pattern Recognition Letters 33 (2012) 559–567
if jPj > 1 then
TjðpÞ ( krRj ðpÞ � rG
j ðpÞk þ krGj ðpÞ � rB
j ðpÞk þ krBj ðpÞ � rR
j ðpÞk
end ifend forif jPj > 1 then
TðpÞ ( 1J
PJj¼1TjðpÞ
end ifend forif jPj > 1 then
for all p such that p 2 P doif TðpÞ > Threshold then
P � P � {p}
g. 1. Comparison of Algorithm 1 with the existing methods: (a)–(d) source imagesparated by JADE (ISR = 0.77), (m)–(p) layers separated by Relative Newton (ISR = 0.19
end ifend for
end ifuntil jPj = 1 and convergence
I1 ( IR1ðpÞI
G1ðpÞI
B1ðpÞ
h i; . . .
; IK ( IRKðpÞI
GK ðpÞI
BKðpÞ
h i; where p 2 P return I1, . . . , IK
4. Results
4.1. Artificial mixtures
We present experiments with synthetic mixtures of four knownimages. Fig. 4(a)–(d) show the source images. We mixed them
, (e)–(h) mixed images (condition number of mixing matrix = 5.37), (i)–(l) layers), (q)–(t) layers separated by our Algorithm 1 (ISR = 0.13).
Fig. 2. Examples of failed image separation under a permutation: (a)–(d) JADE, (e)–(h) Relative Newton, (i)–(l) our Algorithm 1.
Mixed image sequence
Estimated source image sequence
R G B
Three concatenated gray image sequence
Color image sequence
Fig. 3. Image separation through iterative image updates.
K. Hara et al. / Pattern Recognition Letters 33 (2012) 559–567 563
using different mixing ratios, as shown in Fig. 4(e)–(h). We showthe results of our method outlined in Algorithm 2 in Fig. 4(i)–(l).One can see that our approach gives fairly good results.
Fig. 5 shows the process of search and selection of the updatepermutation in this image separation. The vertical axis representsTðpÞ value (Eq. (10)). The horizontal axis represents the numberof cycles executed for the mixture separation and selection of theupdate permutation. According to the criterion for accepting orrejecting update premutation candidates in Algorithm 2, six se-quences {(3,1,4,2),(3,4,1,2),(4,3,2,1),(4,1,3,2),(1,3,4,2),(1,4,3,2)},two sequences {(1,3,4,2), (1,4,3,2)}, and one sequence {(1,4,3,2)}
are selected as candidates after the first, second and third cycles,respectively, as shown by real lines in Fig. 5. Here the dottedlines represent the mixture separation processes for the rejectedpermutations. The labels 1, 2, 3 and 4 represent the images inFig. 4(e), (f), (g) and (h), respectively. The circles and black crossesdepict the update permutations which give perceptionally correctand incorrect separation results, respectively. We confirm thatthe finally selected permutation (1,4,3,2) (drawn as a doublecircle) gives a perceptionally correct separation result, as shownat the right bottom of Fig. 5 (or Fig. 4(m)–(p)). The picture at theright top of Fig. 5 shows an unsuccessful result obtained for one
Fig. 4. Image separation with our Algorithm 2: (a)–(d) source images, (e)–(h) mixed images, (i)–(l) layers separated by our method.
reject reject
reject
failure
success
Fig. 5. Process of mixture separation and selection of the update permutation (artificial mixtures).
564 K. Hara et al. / Pattern Recognition Letters 33 (2012) 559–567
of the rejected permutations. Table 2 compares the success ratiobetween Algorithm 1 and 2. In this experiment, we use a dataset,where each data element (each of No.1–No.10 in the top row inTable 2) contains four image mixtures synthesized using differentmixing ratios. First we visually test Algorithm 1 for every permuta-tion for each data element and then give the (number of successfulpermutations)/(total number of permutations) ratio in the secondrow in Table 2. Next we visually test Algorithm 2 for the same dataelement and then show success or failure of the image separationin the bottom row in Table 2. The success rates of Algorithm 1 and2 are calculated from the above results and are shown in the rightcolumn in Table 2. Among the two algorithms, Algorithm 2 seemsto achieve higher success ratio.
4.2. Real mixtures
We apply our method to layer extraction from images contain-ing multiple reflections and transparency. We photographed a pic-ture postcard in a glass-fronted bookcase (Fig. 6(a)). One can seetransparency and double reflections due to light reflected fromboth the surfaces of the front side and rear side glasses (region sur-rounded by a rectangle of Fig. 6(a)). We took three photographs un-der three different illumination conditions by inserting apolarization sheet at the front or back of two glasses on the book-case, or between them (Fig. 6(c)–(e)). For acquisition of a groundtruth transparency image we shot the same scene while shieldingout some of the ambient light using a blackout curtain (Fig. 6(b)).
Table 2Comparison between Algorithm 1 and 2.
Methods Input data number Success ratio (%)
No. 1 No. 2 No. 3 No. 4 No. 5 No. 6 No. 7 No. 8 No. 9 No. 10
Algorithm 1 number of successful permutationstotal number of permutations
� �12/24 24/24 19/24 7/24 16/24 6/24 23/24 21/24 23/24 2/24 63.5
Algorithm 2 (success: s; failure: �) s s s s s s s s s � 90.0
Fig. 6. Separation of layers from images containing multiple reflections andtransparency: (a) image region containing multiple reflections and transparency,(b) transparency image obtained by shielding out some of the ambient light using ablackout curtain, (c)–(e) three images taken under three different illuminationconditions, (f)–(h) layers separated by JADE, (i)–(k) layers separated by RelativeNewton, (l)–(n) layers separated by our method.
K. Hara et al. / Pattern Recognition Letters 33 (2012) 559–567 565
The separation results obtained by the JADE (Cardoso andSouloumiac, 1993), Relative Newton method (Gribonval andZibulevsky, 2010) and our method are shown in Fig. 6(f)–(h), (i)–(k)and (l)–(n), respectively. One can see that, for both RelativeNewton method and our method, the estimated transparencyimages (Fig. 6(i) and (l)) are similar to the ground truth transpar-ency image (Fig. 6(b)), and that Fig. 6(j), (k), (m) and (n) are alsorelatively clear compared to JADE (Fig. 6(f)–(h)).
In the same way as in the previous case, the process of searchand selection of the permutation in this experiment is shown inFig. 7. Here the labels 1, 2 and 3 represent the images in Fig. 6(c),(d) and (e), respectively. The update permutation (2,3,1) is se-lected and it gives a correct separation result, as shown at the cen-ter of Fig. 7 (or Fig. 6(l)–(n)). The picture at the right top of Fig. 7shows an unsuccessful result obtained for one of the rejectedpermutations.
5. Conclusions
In this paper, we have proposed a method of stably recovering aset of the original source images from the input mixtures. In manyexisting blind image separation techniques the success of the sep-aration depends upon the selection of permutations correspondingto the orders of the update operations. By extending Sarel et al.’sbivariate generalized correlation measure (Sarel and Irani, 2004)to more than two variables and then introducing the simple prun-ing technique based on the discrepancy among the separationparameter values for each of the three RGB channels, we havepresented a image separation method without permutationambiguity. We have presented the separation algorithm, andshown its effectiveness through experiments with mixtures of realimages.
Acknowledgements
This research was partially supported by the Suzuki Foundation.
Appendix A. Derivation of Eq. (5)
In this appendix, we derive Eq. (5) in detail. First we divide thecorrelation matrix Ri
K (Eq. (4)) into four blocks as,
ðA:1Þ
where RiK�1 is a K � 1 dimensional square matrix and ri
K�1 is a K � 1dimensional vector. Hence, from the definition of the inverse ma-trix, ~ri
K;K included in Eq. (3) is expressed as
~riK;K ¼
det RiK�1
� �det Ri
K
� � ; ðA:2Þ
where det (�) denotes the matrix determinant. By first assumingdetðRi
K�1Þ– 0 and then applying the formula (Chen and Gu, 2000)for the determinant of a block matrix to Eq. (A.1), we have
failure
success
Fig. 7. Process of mixture separation and selection of the update permutation (real mixtures).
566 K. Hara et al. / Pattern Recognition Letters 33 (2012) 559–567
det RiK
� �¼ det Ri
K�1
� �� 1� ðri
K�1ÞT Ri
K�1
� ��1ri
K�1
� �; ðA:3Þ
where (�)�1 denotes the inverse matrix. Using Eq. (A.3), we canrewrite Eq. (A.2) as,
~riK;K ¼
1
1� ðriK�1Þ
T RiK�1
� ��1ri
K�1
: ðA:4Þ
Hence, using Eq. (A.4), we can rewrite Eq. (3) as,
MNGCiKðf1; . . . ; fK�1; fKÞ ¼ ðri
K�1ÞT Ri
K�1
� ��1ri
K�1
� �1=2
: ðA:5Þ
Replacing {f1, . . . , fK�1, fK} in Eqs. (2) and (A.5) with IV1 ; . . . ; IV
K�1;eIV
n oand then combining these equations as,
MGNGCK IV1 ; . . . ; IV
K�1;eIV� �
¼PN
i¼1 MNGCiK IV
1 ; . . . ; IVK�1;eIV
� �h i2ViðeIV Þ
QK�1k¼1 Vi IV
k
� �PN
i¼1ViðeIV ÞQK�1
k¼1 Vi IVk
� �¼PN
i¼1 uiK�1
� T U iK�1
� ��1ui
K�1
QK�1k¼1 Vi IV
k
� �PN
i¼1ViðeIV ÞQK�1
k¼1 Vi IVk
� � ; ðA:6Þ
where U iK�1 is the K � 1 dimensional square correlation matrix
whose klth entry is U iK�1
� �kl¼ Ci IV
k ;IVlð Þ
Vi IVkð ÞVi IV
lð Þ½ �1=2 and uiK�1 is the K � 1
dimensional vector whose kth entry is uiK�1
� k ¼
Ci IVk ;eIV
� Vi IV
kð Þ½ �1=2, as
described in Section 2.3.
Appendix B. Derivation of Eq. (6)
Denoting the numerator of Eq. (5) byPN
i¼1Wi as,
Wi ¼ uiK�1
� TU i
K�1
� ��1ui
K�1
YK�1
k¼1
Vi IVk
� �ðB:1Þ
and then taking partial derivatives of Wi with respect to rk, fork = 1, . . . ,K � 1, we get
@Wi@rk¼ @ui
K�1@rk
� �T
� @Wi@ui
K�1
¼ �2QK�1
l¼1ViðIV
l ÞCi IV
1 ;IVkð Þ
Vi IV1ð Þ½ �1=2 � � �
Ci IVK�1 ;I
Vkð Þ
Vi IVK�1ð Þ½ �1=2
� �
�ðU iK�1Þ
�1 Ci IV1 ;eIV
� Vi IV
1ð Þ½ �1=2 � � �Ci IV
K�1 ;eIV
� Vi IV
K�1ð Þ½ �1=2
� �T
¼ 2QK�1
l¼1Vi IV
l
� �Vi IV
k
� �h i1=2ui
1;k � � �uiK�1;k
h i� U i
K�1
� ��1 Ci IV1 ;eIV
� Vi IV
1ð Þ½ �1=2 � � �Ci IV
K�1 ;eIV
� Vi IV
K�1ð Þ½ �1=2
� �T
¼ 2QK�1
l¼1Vi IV
l
� �0 � � �0 Vi IV
k
� �h i1=20 � � �0
� �
� Ci IV1 ;eIV
� Vi IV
1ð Þ½ �1=2 � � �Ci IV
K�1 ;eIV
� Vi IV
K�1ð Þ½ �1=2
� �T
¼ 2CieIV ; IV
k
� � QK�1
l¼1Vi IV
l
� �¼ 2 Ci IV
K ; IVk
� ��PK�1
m¼1rmCi IV
m; IVk
� �� � QK�1
l¼1Vi IV
l
� �:
ðB:2Þ
So, the partial derivatives of the numerator,PN
i¼1Wi, of Eq. (5) withrespect to rk, for k = 1, . . . ,K � 1, are given by
@
@rk
XN
i¼1
Wi ¼ 2XN
i¼1
Ci IVk ; I
VK
� ��XK�1
m¼1
rmCi IVk ; I
Vm
� �" #YK�1
l¼1
Vi IVl
� �ðk ¼ 1; . . . ;K � 1Þ: ðB:3Þ
Since one can easily see that for k = 1, . . . ,K � 1, the partial deriva-
tives of the denominator,PN
i¼1ViðeIV ÞQK�1
k¼1 Vi IVk
� �, of Eq. (5) with re-
spect to rk is also equal to Eq. (B.3), we have
@F@rk¼ @G@rk¼ 2
XN
i¼1
Ci IVk ; I
VK
� ��XK�1
m¼1
rmCi IVk ; I
Vm
� �" #YK�1
l¼1
Vi IVl
� �ðk ¼ 1; . . . ;K � 1Þ; ðB:4Þ
where F and G are the denominator and the numerator of Eq. (5),respectively.
Thus, taking partial derivatives of (5) with respect to rk and set-ting them to zero, we get
K. Hara et al. / Pattern Recognition Letters 33 (2012) 559–567 567
0 ¼ @
@rkMGNGCK IV
1 ; . . . ; IVK�1;eIV
� �¼ @
@rk
FG
� �¼ 1
G2 G@F@rk� F
@G@rk
� �¼ 1
G1� F
G
� �@F@rk
ðk ¼ 1; . . . ;K � 1Þ: ðB:5Þ
Then, from the definition of MGNGC and the independency offIV
k gKk¼1, we have F/G < 1 Hence, we get @ F/@rk = 0 in (B.5), which
leads to the following equation.
@F@rk¼ 2
XN
i¼1
Ci IVk ; I
VK
� ��XK�1
m¼1
rmCi IVk ; I
Vm
� �" #YK�1
l¼1
Vi IVl
� �ðk ¼ 1; . . . ;K � 1Þ: ðB:6Þ
This equation can be rewritten as,XK�1
m¼1
XN
i¼1
Ci IVk ; I
Vm
� �YK�1
l¼1
Vi IVl
� �" #rV
m
¼XN
i¼1
Ci IVk ; I
VK
� �YK�1
l¼1
Vi IVl
� �ðk ¼ 1; . . . ;K � 1Þ: ðB:7Þ
References
Bedini, L., Herranz, D., Salerno, E., Baccigalupi, C., Kuruoglu, E., Tonazzini, A., 2005.Separation of correlated astrophysical sources using multiple-lag datacovariance matrices. EURASIP J. Appl. Sig. Proc. 15, 2400–2412.
Bell, A.J., Sejnowski, T.J., 1995. An information maximization approach to blindseparation and blind deconvolution. Neural Comput. 7 (6), 1129–1159.
Bronstein, A.M., Bronstein, M.M., Zibulevsky, M., Zeevi, Y.Y., 2005. Sparse ICA forblind separation of transmitted and reflected images. Internat. J. Imag. Sci.Technol. 15 (1), 84–91.
Cardoso, J.F., Souloumiac, A., 1993. Blind beamforming for non gaussian signals. In:IEE Proceedings-F, 140(6), 362–370.
Cardoso, J.F., 1994. On the performance of orthogonal source separation algorithms.In: Proc. EUSIPCO, pp. 776–779.
Cardoso, J.F., Snoussi, H.H., Delabrouille, J., Patanchon, G., 2002. Blind separation ofnoisy gaussian stationary sources: Application to cosmic microwave imaging.In: Proc. EUSIPCO, vol. 1, pp. 561–564.
Chen, J., Gu, G., 2000. Control-Oriented System Identification: An H-InfinityApproach. John Wiley.
Cichocki, A., Amari, S., 2002. Adaptive Blind Signal and Image Processing. JohnWiley.
Diamantaras, K.I., Papadimitriou, T., 2005. Blind separation of reflections using theimage mixtures ratio. In: Proc. IEEE ICIP, pp. 1034–1037.
Farid, H., Adelson, E.H., 1999. Separating reflections from images by use ofindependent components analysis. J. Opt. Soc. America A 16 (9), 2136–2145.
Gribonval, R., Zibulevsky, M., 2010. Sparse component analysis. In: Comon, Pierre,Jutten, Christian (Eds.), Handbook of Blind Source Separation: IndependentComponent Analysis and Applications. ELSEVIER.
Hara, K., Inoue, K., Urahama, K., 2009. Separation of layers from images containingmultiple reflections and transparency using cyclic permutation. In: Proc. IEEEICASSP, pp. 1157–1160.
Hyvaerinen, A., Oja, E., 1997. A fast fixed-point algorithm for independentcomponent analysis. Neural Comput. 9 (7), 1483–1492.
Kayabol, K., Kuruoglu, E.E., Sankur, B., 2009. Image source separation using colorchannel dependencies. In: Proc. ICA, pp. 499–506.
Kopriva, L., 2007. Approach to blind image deconvolution by multiscale subbanddecomposition and independent component analysis. J. Opt. Soc. America A 24(4), 973–983.
Levin, A., Weiss, Y., 2007. User assisted separation of reflections from a single imageusing a sparsity prior. IEEE Trans. Pattern. Anal. Machine. Intell. 29 (9), 1647–1654.
Levin, A., Zomet, A., Weiss, Y., 2004. Separating reflections from a single image usinglocal features. In: Proc. IEEE CVPR, pp. 306–313.
Lin, S., Shum, H.Y., 2001.Separation of diffuse and specular reflection in colorimages. In: Proc. IEEE CVPR, pp. 341–346.
Nayar, S.K., Fang, X.S., Boult, T., 1996. Separation of reflection components usingcolor and polarization. Internat J. Comput. Vision. 21 (3), 163–186.
Sarel, B., Irani, M., 2004. Separating transparent layers through layer informationexchange. In: Proc. ECCV, pp. 328–341.
Tonazzini, A., Bedini, L., Salerno, E., 2006. A markov model for blind imageseparation by a mean-field EM algorithm. IEEE Trans. IP. 15 (2), 473–482.
