Multichannel Color Image Denoising via Weighted Schatten p ... · challenges faced by RGB color image denoising. Intuitive-ly, if we apply the grayscale image denoising algorithm

Multichannel Color Image Denoising via Weighted Schattenp-norm Minimization

Xinjian Huang , Bo Du∗ and Weiwei Liu∗

School of Computer Science, Institute of Artificial Intelligence andNational Engineering Research Center for Multimedia Software, Wuhan University, Wuhan, China

h [email protected], [email protected], [email protected]

AbstractThe R, G and B channels of a color image gen-erally have different noise statistical properties ornoise strengths. It is thus problematic to applygrayscale image denoising algorithms to color im-age denoising. In this paper, based on the non-localself-similarity of an image and the different noisestrength across each channel, we propose a Multi-Channel Weighted Schatten p-Norm Minimization(MCWSNM) model for RGB color image denois-ing. More specifically, considering a small localRGB patch in a noisy image, we first find its nonlo-cal similar cubic patches in a search window withan appropriate size. These similar cubic patches arethen vectorized and grouped to construct a noisylow-rank matrix, which can be recovered using theSchatten p-norm minimization framework. More-over, a weight matrix is introduced to balance eachchannel’s contribution to the final denoising result-s. The proposed MCWSNM can be solved via thealternating direction method of multipliers. Con-vergence property of the proposed method are alsotheoretically analyzed . Experiments conducted onboth synthetic and real noisy color image dataset-s demonstrate highly competitive denoising perfor-mance, outperforming comparison algorithms, in-cluding several methods based on neural networks.

1 IntroductionNoise corruption is inevitable during the image acquisitionprocess and may heavily degrade the visual quality of an ac-quired image. Image denoising is thus an essential prepro-cessing step in various image processing and computer vi-sion tasks [Chatterjee and Milanfar, 2010; Ye et al., 2018;Wang et al., 2018b]; moreover, it is also an ideal test platformfor evaluating image prior models and optimization methods[Roth and Black, 2005]. As a result, image denoising re-mains a challenging yet fundamental problem. Early denois-ing algorithms were mainly devised on the basis of filter andtransformation [Dabov et al., 2007b], such as wavelet trans-form and curvelet transform [Starck et al., 2002]. State-of-

∗Co-corresponding Author.

the-art denoising methods are mainly based on sparse repre-sentation [Dabov et al., 2007b], low-rank approximation [Guet al., 2017; Xie et al., 2016], dictionary learning [Zhang andAeron, 2016; Marsousi et al., 2014; Mairal et al., 2012], non-local self-similarity [Buades et al., 2005; Dong et al., 2013a;Hu et al., 2019] and neural networks [Liu et al., 2018;Zhang et al., 2017a; Zhang et al., 2017b; Zhang et al., 2018].

Current methods for RGB color image denoising can becategorized into three classes. The first kind involves apply-ing grayscale image denoising algorithms to each channel ina channel-wise manner. However, these methods ignore thecorrelations between R, G and B channels, meaning that un-satisfactory results may be obtained. The second method isto transform the RGB color image into other color spaces [D-abov et al., 2007a]. This transform, however, may change thenoise distribution of the original observation data and intro-duce artifacts. The third type of method involves making fulluse of the correlation information across each channel andconduct the denoising task on R, G and B channels simulta-neously [Zhang et al., 2017a].

Similar to the case of grayscale images, noise from eachchannel can be generally, regarded as additive white Gaussiannoise. However, the noise levels of each channel are diversedue to camera sensor characteristics and the imagery environ-ment, such as fog, haze, illumination intensity, etc. Moreover,the on-board processing steps in digital camera pipelines mayalso introduce different noise strength assigned to differentchannels [Nam et al., 2016]. This reveals the difficulties andchallenges faced by RGB color image denoising. Intuitive-ly, if we apply the grayscale image denoising algorithm tocolor images in a band-wise manner without considering themutual information and noise difference between each chan-nel, artifacts or false colors could be generated [Mairal et al.,2008]. An example of this is presented in Fig.1. Here, weapply a representative low-rank-based method, WSNM [Xieet al., 2016], and our proposed method, MCWSNM (see Sec-tion 2), to the Kodak PhotoCD Dataset1. It can be seen fromthe figure that WSNM retains a large amount of noise and,to some extent, introduces some artifacts. Thus, the issue ofhow noise differences in each channel should be modeled iskey to designing a good RGB color denoising algorithm.

One well-known color image denoising method is color

1http://r0k.us/graphics/kodak/

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20)

637

(a) Clean (b) Noisy (c) WSNM (d) MCWSNM

Figure 1: Denoised results of ”kodim07” in the Kodak PhotoCDDataset with σr = 50, σg = 5 and σb = 20. WSNM performsin a band-wise manner; MCWSNM jointly processes three channelsjointly and considers their noise differences simultaneously. (It isbest to zoom in on these images on-screen).

block-matching 3D (CBM3D) [Dabov et al., 2007a]. Thisis a transformation field method that may destroy the noisedistribution across each channel and introduce artifacts. Themethod in [Zhu et al., 2016] concatenates similar patches ofRGB channels into a long vector. However, this concatena-tion operation regards each channel equally and ignores thenoise level differences across each channel. For a long time,low-rank theory has been widely used in many fields, suchas recommendation system [Wang et al., 2018a], communitysearch [Fang et al., 2020b; Fang et al., 2020a], multi-out task[Liu et al., 2019] and multi-label[Liu et al., 2017], etc.

Recently, based on the non-local self-similarity of an imageand SVD, several low-rank-based denoising algorithms havebeen proposed, such as weighted nuclear norm minimization(WNNM) [Gu et al., 2017] and WSNM [Xie et al., 2016].The nuclear norm and Schatten p-norm [Xie et al., 2016] aretwo surrogates of the rank function [Xu et al., 2017a]. InWNNM and WSNM, singular values are assigned differentweights using a weight vector. It has been proven that WNN-M is a special case of WSNM and that WSNM outperform-s WNNM. Multi-channel WNNM (MCWNNM) [Xu et al.,2017b] is an extension of WNNM from grayscale image tocolor image that operates by introducing a weight matrix toadjust each channel’s contribution to the final results basedon noise level.

In this paper, based on the low-rank property of the non-local self-similar patches and the different noise strengthacross each channel, we propose a multi-channel weightedSchatten p-norm minimization (MCWSNM) model for RG-B color image denoising. More specifically, considering asmall local RGB patch in a noisy image, we first find its non-local similar cubic patches in a search window with an appro-priate size. These similar cubic patches are then vectorizedand concatenated to construct a noisy low-rank matrix; thenSchatten p-norm minimization method is performed to recov-er the noise-free low-rank matrix. Moreover, in order to makefull use of the redundant information across each channel, aweight matrix is introduced to balance each channel’s con-tribution to the final denoising results. This model can besolved via alternating direction method of multipliers (AD-MM) framework, and each variable can be updated with aclosed-form solution. In addition, the convergence propertyof our proposed algorithm is also provided.

rgb

…

…

1

2

M

3…

1

M

Figure 2: Illusion of the grouped local patches.

2 Our Proposed Method2.1 Method FormulationThe image denoising task involves recovering a clean imagexc from its noisy observation data yc, where c = {r, g, b}is the index of the R, G and B channels. Here, we assumethat the noisy image is corrupted by additive white Gaussiannoise nc with σc = (σr, σg, σb), where σr, σg, σb denote thenoise standard deviation in the R, G, B channels, respectively.Mathematically, yc = xc + nc. Low-rank-based denoisingmethods utilize the low-rank property of the grouped non-local similarity patches; this group procedure is illustrated inFig.2. Given a noisy RGB color image yc, each local patchof size s × s × 3 is extracted and stretched into a vectory = (yTr ,y

Tg ,y

Tb )T ∈ R3s2 , where yr,yg,yb ∈ Rs2 are

the corresponding patches in the R, G and B channels. Foreach vectorized local patch y, we find its M most similarpatches (including y itself) by Euclidean distance in a propersize search window around it. We then stack the M similarpatches vector column by column to obtain the noisy patchmatrix Y = X + N ∈ R3s2×M , where X and N are thecorresponding clean and noise patch matrix, respectively. Itis clear that Y is a low-rank matrix and X can be recoveredusing a low-rank approximation algorithm.

As a nonconvex surrogate of the rank function, the weight-ed Schatten p-norm of a matrix Z ∈ Rm×n is defined as‖Z‖w,Sp

= (∑min{m,n}i=1 wiσ

pi )

1p , 0 < p < 1, where w =

(w1, w2, . . . , wmin{m,n})T is a non-negative weight vector

and σi is the i-th singular value of Z. The weighted nuclearnorm [Gu et al., 2017] is a special case of the weighted Schat-ten p-norm when the power p = 1. It has been proven that theSchatten p-norm with 0 < p < 1 is a better approximation ofrank function than the nuclear norm [Zhang et al., 2013].

Intuitively, when denoising a noisy color image, if a chan-nel is heavily polluted, the contribution of this channel to thefinal denoised results should be less, and vice versa. Thus,a weight matrix W assigned to the noise strength of eachchannel is introduced. Based on the low-rank property ofthe non-local self-similarity, we propose the following multi-channel WSNM (MCWSNM) model to recover the local low-rank patch X:

minX‖W (Y −X)‖2F + λ‖X‖pw,Sp

, (1)

where λ > 0 is a tradeoff parameter used to balance the


638

Algorithm 1 minX ‖Y −X‖2F + λ‖X‖pw,Spvia GST

Input: Y , weight {wi}ri=1, λ, p,K1: Singular value decomposition Y = UΣV T , Σ =diag{σ1, σ2, . . . , σr};

2: for i = 1 : r do3: τGSTp (wi, λ) = (2λwi(1− p))

12−p + λwip(2λwi(1−

p))p−12−p ;

4: if |σi| ≤ τGSTp (wi) then5: δi = 0;6: else7: δ

(k)i = |σi|;

8: for k = 0, 1, . . . ,K do9: δ

(k+1)i = |σi| − wip(δ(k)i )p−1;

10: k = k + 1;11: end for12: δi = sgn(σi)δ

(k)i ;

13: end if14: end for15: ∆ = diag(δ1, δ2, . . . , δr);Output: X∗ = U∆V T

data fidelity term and the regularization term, while W =

diag(τrI, τgI, τbI), I ∈ Rs2×s2 is the identity matrix,(τr, τg, τb) = min{σr, σg, σb}/(σr, σg, σb).

2.2 Model OptimizationWe utilize the variable splitting method to the MCWSNMmodel (1). By introducing an augmented variable Z, model(1) can be reformulated as a linear constrained optimizationproblem. Thus, model (1) can be expressed as follows:

minX,Z

‖W (Y −X)‖2F + λ‖Z‖pw,Sp, s.t.X = Z. (2)

Because the two variables X and Z in problem (2) areseparable, this programming problem can be solved using theADMM framework. The augmented Lagrangian function of(2) is

L(X,Z,L, ρ) = ‖W (Y −X)‖2F + λ‖Z‖pw,Sp

+ Tr(LT (X −Z)) +ρ

2‖X −Z‖2F , (3)

where L is the augmented Lagrangian multiplier and ρ > 0 isthe penalty parameter. By taking derivatives ofLwith respectto X and Z and setting the derivative function to be zero, thevariables can be alternatively updated as follows:(1) Xk+1

= (W TW +ρk2I)−1(W TWY +

ρk2Zk −

1

2Lk). (4)

(2) Zk+1

= argminZ

ρk2‖Z − (Xk+1 +

1

ρkLk)‖2F+λ‖Z‖

pw,Sp

. (5)

This is a weighted Schatten p-norm minimization problem.It was noted in [Xie et al., 2016] that if the weights follow thenon-descending permutation, (5) can be equivalently trans-formed into independent nonconvex `p-norm subproblems,

Algorithm 2 Solve MCWSNM via ADMM

Input: data Y , weight W , p, K1, µ > 1, tol > 0;1: Initialization: X0 = Z0 = 0, L0 = 0, ρ0 > 0, flag =

False, k = 0;2: while flag == False do3: Update X by using (4);4: Update Z by solving (5);5: Update L: Lk+1 = Lk + ρk(Xk+1 −Zk+1);6: Update ρ: ρk+1 = µρk;7: k = k + 1;8: if (Convergence condition is satisfied) or (k ≥ K1)

then9: flag = True;

10: end if11: end whileOutput: X∗.

the global optima of which can be efficiently solved by thegeneralized soft-thresholding (GST) algorithm. This is sum-marized in Algorithm 1.(3) Lk+1 = Lk + ρk(Xk+1 −Zk+1).(4) ρk+1 = µρk, (µ > 1).

The above alternative updating steps are repeated until ei-ther the convergence condition is satisfied or the number ofiterations exceeds a preset K1. The ADMM algorithm con-verges when 1) ‖Xk+1 − Zk+1‖F ≤ tol, 2) ‖Xk+1 −Xk‖F ≤ tol and 3) ‖Zk+1−Zk‖F ≤ tol are simultaneouslysatisfied; here, tol is a small tolerance value. The completeupdating procedures are summarized in Algorithm 2.

2.3 Convergence and ComplexityAlthough the proposed MCWSNM (1) is nonconvex, theglobal optimum also can be obtained for the limitations ofweights permutation in GST. From Fig.3, it can be seen that‖Xk+1 −Xk‖2, ‖Zk+1 − Zk‖2 and ‖Xk+1 − Zk+1‖2 si-multaneously approach 0 during the iteration process. Thiscurve is based on a synthetic experiment on Kodak PhotoCDDataset and the test image is ”kodim01” (see ExperimentsSection in detail).

0 5 10 15 20 25 30 35 40 45 50iteration num.

0

20

40

60

80

100

120

140

160

180

valu

e

||Xk+1-Xk||2||Xk+1-Zk+1||2||Zk+1-Zk||2

Figure 3: The convergence curves of ‖Xk+1 − Xk‖2, ‖Zk+1 −Zk‖2 and ‖Xk+1 − Zk+1‖2 of image ”kodim01” in the KodakDataset.


639

We next provide a theorem that theoretically guarantees theconvergence property of Algorithm 2.

Theorem 1. Assume that the weights w are sorted in anon-descending order and the parameter ρk is unbounded;then the sequences {Xk}, {Zk} and {Lk} generated in Al-gorithm 2 satisfy: (a) limk→+∞ ‖Xk+1 − Zk+1‖F = 0;(b) limk→+∞ ‖Xk+1 −Xk‖F = 0; (c) limk→+∞ ‖Zk+1 −Zk‖F = 0.

Proof. We first prove that the sequence {Lk} generated byAlgorithm 2 is upper bounded. UkΛkV

Tk denotes the SVD

of matrix 1ρkLk + Xk+1 in the k + 1 iteration, where Λk

is the diagonal singular value matrix. By using the GST al-gorithm for WSNM, we have Zk+1 = Uk∆kV

Tk , where

∆k = {diag(δ1k, δ2k, . . . , δnk )} is the diagonal singular valuematrix after generalized soft-thresholding operation. Then:

‖Lk+1‖2F = Lk + ρk(Xk+1 −Zk+1)‖2F

= ρ2k‖1

ρkLk +Xk+1 −Zk+1‖2F

= ρ2k‖UkΛkVTk −Uk∆kV

Tk ‖2F = ρ2k‖Λk −∆k‖2F

= ρ2k‖∑i

rwi/ρk‖2F = ‖r∑i

wi‖2F .

Hence, the sequence {Lk} is upper bounded.We then prove that the sequence of the Lagrange function

{L(Xk+1,Zk+1,Lk+1, ρk+1)} is also upper bounded. Theinequality L(Xk+1,Zk+1,Lk, ρk) ≤ L(Xk,Zk,Lk, ρk) al-ways holds since we have the globally optimal solution of Xand Z in their corresponding subproblems (step 3 and step 4in Algorithm 2. Based on the updating rule of L, it yields

L(Xk+1,Zk+1,Lk+1, ρk+1)

= ‖W (Y −Xk+1)‖2F + ‖Zk+1‖pw,Sp

+ 〈Lk+1,Xk+1 −Zk+1〉+ρk+1

2‖Xk+1 −Zk+1‖2F

= L(Xk+1,Zk+1,Lk, ρk) + 〈Lk+1 −Lk,Xk+1 −Zk+1〉

+ρk+1 − ρk

2‖Xk+1 −Zk+1‖2F

= L(Xk+1,Zk+1,Lk, ρk) + 〈Lk+1 −Lk,Lk+1 −Lk

ρk〉

+ρk+1 − ρk

2‖Lk+1 −Lk

ρk‖2F

= L(Xk+1,Zk+1,Lk, ρk) +ρk+1 + ρk

2ρ2k‖Lk+1 −Lk‖2F .

Since {Lk} is upper bounded, the sequence {Lk+1 −Lk} isalso upper bounded. Denote by a the upper bound of {Lk+1−Lk} for all k ≥ 0, i.e. {Lk+1 − Lk} ≤ a, ∀k ≥ 0. Wetherefore conclude that

L(Xk+1,Zk+1,Lk+1, ρk+1)

≤ L(Xk+1,Zk+1,Lk, ρk) +ρk+1 + ρk

2ρ2ka2

≤ L(X1,Z1,L0, ρ0) + a2∑∞k=0

ρk+1+ρk2ρ2k

= L(X1,Z1,L0, ρ0) + a2∑∞k=0

1+µ2ρ0µk

≤ L(X1,Z1,L0, ρ0) +a2

ρ0

∑∞k=0

1µk−1

< +∞.

Thus, {L(Xk+1,Zk+1,Lk+1, ρk+1)} is upper bounded.We next prove the sequences of {Xk} and {Zk} are upper

bounded. Since {L(Xk,Zk,Lk, ρk)} and {Lk} are upperbounded and

‖W (Y −Xk)‖2F + ‖Zk‖pw,Sp

= L(Xk,Zk,Lk−1, ρk−1)− 〈Lk−1,Xk −Zk〉

− ρk−12‖Xk −Zk‖2F

= L(Xk,Zk,Lk−1, ρk−1)− 〈Lk−1, (Lk −Lk−1)/ρk−1〉

− ρk−12‖Lk −Lk−1

ρk−1‖2F

= L(Xk,Zk,Lk−1, ρk−1) +‖Lk−1‖2F − ‖Lk‖2F

2ρk−1,

and the sequence {W (Y −Xk)} and {Zk} are upper bound-ed. Since Lk+1 = Lk + ρk(Xk+1 − Zk+1), {Xk} is alsoupper bounded. Thus, there exists at least one accumulationpoint for {Xk,Zk}. More specifically, we obtain that

limk→+∞

‖Xk+1 −Zk+1‖F = limk→+∞

1

ρk‖Lk+1 −Lk‖F = 0

and that the accumulation point is a feasible solution to theobjective function. Thus, equation (a) is proved.

Finally, we prove that the change of sequence {Xk} and{Zk} in adjacent iterations tends to be 0. For Xk+1 andXk = 1

ρk−1(Lk −Lk−1) +Zk, we have

limk→∞

‖Xk+1 −Xk‖F

= limk→∞

‖(W TW +ρk2I)−1(W TWY +

ρk2Zk −

1

2Lk)

− 1

ρk−1(Lk −Lk−1)−Zk‖F

= limk→∞

‖(W TW +ρk2I)−1(W TWY −W TWZk

− 1

2Lk)−

1

ρk−1(Lk −Lk−1)‖F

≤ limk→∞

‖(W TW +ρk2I)−1(W TWY −W TWZk

− 1

2Lk)‖F +

1

ρk−1‖(Lk −Lk−1)‖F

= 0.


640

Since Zk+1 = 1ρkLk − 1

ρkLk+1 +Xk+1, we conclude that

limk→∞

‖Zk+1 −Zk‖F

= limk→∞

‖ 1ρk

Lk −1

ρkLk+1 +Xk+1 −Zk‖F

= limk→∞

‖Xk +1

ρk−1Lk−1 −Zk +Xk+1

−Xk −1

ρk−1Lk−1 +

1

ρkLk −

1

ρkLk+1‖F

≤ limk→∞

‖∑i

rwiρk−1

‖F + ‖Xk+1 −Xk‖

+ ‖ 1

ρk−1Lk−1 −

1

ρkLk +

1

ρkLk+1‖F = 0.

This completes the proof.

The main computational cost in a single iteration of Algo-rithm 2 consists of two parts. The first part involves updat-ing X , in which the time complexity isO(max{s4M,M3}).The second part involves updating Z. The predominant costof this updating is the SVD and the calculation of X∗. It-s complexity is O(min{s4M, s2M2} + s2r4M). The costsfor updating L and ρ can be ignored. Therefore, the totaltime complexity of our MCWSNM for solving problem (1) isabout O(K1M

3).

3 ExperimentsTo illustrate the performance of the proposed MCWSNM, weimplement MCWSNM on synthetic and real color noisy im-ages. We also compare the proposed method with several re-cent proposed methods, including NC [Lebrun et al., 2015],NCSR [Dong et al., 2013b], PGPD [Xu et al., 2015], M-CWNNM [Xu et al., 2017b], DnCNN [Zhang et al., 2017a],FFDNet [Zhang et al., 2018] and IRCNN [Zhang et al.,2017b]. In more detail, NC is an online blind image denois-ing platform, NCSR and PGPD are designed for grayscaleimages and MCWNNM is a color image denoising methodwhile DnCNN, FFDNet, and IRCNN are three methods thatoperate on the basis of convolutional neural networks. All theparameters of the comparison algorithms are either optimallyassigned, or chosen as described in the reference papers.

Noise level of MCWSNM and most comparative methodsshould be provided as parameters. In the synthetic experi-mental case, the noise (σr, σg, σb) in the R, G and B channelsare assumed to be known. In the case involving real noise,we utilize the noise estimation method outlined in [Chen etal., 2015] to estimate the noise level of the noisy image foreach channel. We implement the grayscale denoising meth-ods, NCSR and PGPD, on the color noisy images in a band-wise manner with the corresponding channel noise level.

3.1 Synthetic Noisy Color Image ExperimentsThe Kodak PhotoCD Dataset is first utilized in our syntheticexperiments. It includes 24 color images, each of which iseither 768 × 512 or 512 × 768 in size. The noisy image isgenerated by adding zero mean Gaussian noise with σr = 40,σg = 20 and σb = 30. In MCWSNM, we set the local search

5 10 15 20Image Number

23

24

25

26

27

28

29

30

31

32

PSN

R(d

B)

NC NCSR PGPD MCWNNM DnCNN FFDNet IRCNN MCWSNM

Figure 4: Line graph of the denoising results in Kodak Dataset.

(a) (b) (c) (d) (e)

(f) (g) (h) (i) (j)

Figure 5: Denoised images of ”kodim1” in Kodak PhotoCD Datasetwith σr = 40, σg = 20 and σb = 30. ((a): Original, (b): Noisy,(c): NC, (d): NCSR, (e): PGPD, (f): MCWNNM, (g): DnCNN, (h):FFDNet, (i): IRCNN, (j): MCWSNM).

(a) Original (b) Noisy (c) NC (d) NCSR (e) PGPD

(f) MCWNNM (g) DnCNN (h) FFDNet (i) IRCNN (j) MCWSNM

Figure 6: Denoised images of cropped ”kodim3” in the Kodak Pho-toCD Dataset with σr = 40, σg = 20 and σb = 30.

window size of each patch as 20, the similar patch numberM = 70, each patch size s = 6, K1 = 10, K = 4, c = 2

√2,

λ = 0.6, p = 0.999 and ρ = 3.The line graph of the PSNR results for MCWSNM and the

comparison methods are presented in Fig.4. It can be seenfrom this graph that our proposed method outperforms theother competing methods in most cases. Moreover, Fig.5 andFig.6 show the denoised results of ”kodim1” and ”kodim3”


641

(a) #1 (b) #2 (c) #3 (d) #4 (e) #5

(f) #6 (g) #7 (h) #8 (i) #9 (j) #10

Figure 7: Ten noisy images in the CC Dataset.

(a) GT (b) Noisy (c) NC (d) NCSR (e) PGPD


Figure 8: Denoised images of cropped #7 in real Dataset CC.

in Kodak PhotoCD Dataset, respectively. Visual inspectionreveals that MCWSNM can obtain the best visual result andclear texture information. Color artifacts are generated byNCSR and PGPD, while NC over-smooths the image. WhileMCWNNM, DnCNN, FFDNet and IRCNN can get a cleanimage, some of the detailed information is lost compared withthe original image.

3.2 Real Noisy Color Image ExperimentsWe implement the WCWSNM on a real noisy color imagedataset, CC [Nam et al., 2016], to evaluate the method’s per-formance. This dataset includes 11 static scenes. The noisyimages were captured in an indoor environment with differ-ent cameras and camera settings. For each scene, 500 imageswere taken using the same camera and camera settings. Themean image of each scene was then calculated to generatethe noisy-free images, which were roughly regarded as theground truth (GT). As the size of the original image is verylarge, 60 cropped smaller images with size 512 × 512 wereprovided in [Nam et al., 2016]. In this paper, we select 10cropped images (see Fig.7) to conduct the experiment.

For MCWSNM, we set the local search window size foreach patch as 20, the similar patch number M = 60, eachpatch size s = 5, K1 = 10, K = 4, c = 2

√2, λ = 0.6, p =

0.999 and ρ = 3. Because the GT are provided, a quantitativeassessment of each method is accessible. PSNR results fordifferent cameras and camera settings are provided in Table

(a) GT (b) Noisy (c) NC (d) NCSR (e) PGPD


Figure 9: Denoised images of cropped image #10 in the real DatasetCC.

0.150.250.350.450.550.650.750.85 0.95p

25

26

27

28

29

30

31

32

33

Aver

age

PSN

R (d

B)(a) (10,15,20)

0.150.250.350.450.550.650.750.85 0.95p

15

20

25

30

Aver

age

PSN

R (d

B)

(b) (30,35,40)

0.150.250.350.450.550.650.750.850.95p

16

18

20

22

24

26

Aver

age

PSN

R (d

B)

(c) (40,45,50)

0.150.250.350.450.550.650.750.850.95p

14

16

18

20

22

24

Aver

age

PSN

R (d

B)(d) (50,55,60)

0.150.250.350.450.550.650.750.850.95p

14

16

18

20

22

24

Aver

age

PSN

R (d

B)

(e) (60,65,70)

0.150.250.350.450.550.650.750.85 0.95p

12

14

16

18

20

22

24

Aver

age

PSN

R (d

B)

(f) (80,85,90)

Figure 10: The influence of changing p on denoised results underdifferent noise levels (σr, σg, σb) on 24 images in the Kodak Pho-toCD Dataset.

1. It can be seen from the table that the proposed MCWSNMobtains the highest PSNR value in most noisy images. Fig.8 and Fig. 9 show the visual results of #7 and #10 imagesin CC dataset, respectively. Compared with other methods,MCWSNM gets the best visual effect. It not only remove thenoise completely, but also preserve the detailed informationeffectively.


642

Camera Settings # NC NCSR PGPD MCWNNM DnCNN FFDNET IRCNN MCWSNM

Canon 5D, ISO=3200 1 38.5689 37.8693 36.8979 40.8160 40.8445 40.6137 41.3212 42.12422 35.0985 34.7144 33.8614 36.5158 35.7199 35.6380 36.0826 35.8209

Nikon D600, ISO=3200 3 36.8310 36.4562 35.8916 38.9327 38.9287 38.7748 39.3098 37.94674 38.6322 36.3762 35.9296 40.1002 39.3217 38.7881 40.4201 40.4434

Nikon D800, ISO=1600 5 38.2259 36.8040 36.2972 39.4494 39.8094 39.5719 39.9118 40.04716 39.0023 37.3847 36.4443 42.3815 40.6930 40.2738 42.8365 41.9094

Nikon D800, ISO=3200 7 36.5622 34.2501 33.9318 39.5804 36.7582 36.2557 38.9257 41.55968 35.9500 33.8988 33.5135 36.7548 36.0972 35.8297 38.2784 37.8922

Nikon D600, ISO=6400 9 33.2295 31.1022 30.7797 36.5241 32.1810 31.9962 33.4272 38.252010 32.2609 30.6858 30.4058 32.2105 31.6635 31.4748 32.7171 32.4324

Table 1: PSNR results (dB) of real color image CC Dataset.

3.3 Analysis of Power pIt is necessary to analyze the most appropriate setting of pow-er p for different noise levels (σr, σg, σb). We utilize 24 im-ages in Kodak PhotoCD Dataset to test the proposed MCWS-NM with different p under different noise levels added to theR, G and B channels. The results are presented in Fig.10. Ineach subfigure, the vertical coordinate represents the averagePSNR value under a certain noise level, while the horizon-tal coordinate denotes the values of p changing from 0.1 to1 with interval 0.05. We use six levels in this test: σc ={(10, 15, 20), (30, 35, 40), . . . , (60, 65, 70), (80, 85, 90)}. Itis clear from the histogram that at low and medium noise lev-els, the best value for p is 1, while at a high noise level, thebest value of p is 0.1. This is mainly because, in the strongnoise case (which means more rank components of the dataare contaminated) highly ranked parts should penalized heav-ily, while lower-ranked parts should be penalized less.

4 ConclusionIn this paper, based on the low-rank property of the non-localself-similarity, we propose a MCWSNM method for colorimage denoising. For noisy color images, which generallyhold different noise strength in each band, a weight matrixassigned to the noise level of each channel is introduced inorder balance each channel’s the contribution to the final es-timation result. MCWSNM can be efficiently solved via AD-MM optimization framework. Theorem 1 theoretically an-alyzes the convergence property of our proposed algorithm.Experiments on synthetic and real datasets demonstrate thatthe proposed method can obtain satisfactory results on thecolor image denoising task.

AcknowledgmentsThis work was supported in part by the National Natu-ral Science Foundation of China under Grants 61822113,61976161, 62041105, the Science and Technology MajorProject of Hubei Province (Next-Generation AI Technolo-gies) under Grant 2019AEA170, the Natural Science Founda-tion of Hubei Province under Grants 2018CFA050, the Fun-

damental Research Funds for the Central Universities underGrant 413000092 and 413000082.

References[Buades et al., 2005] Antoni Buades, Bartomeu Coll, and

Jean-Michel Morel. A non-local algorithm for image de-noising. In CVPR, pages 60–65, 2005.

[Chatterjee and Milanfar, 2010] Priyam Chatterjee and Pey-man Milanfar. Is denoising dead? IEEE Transactions onImage Processing, 19(4):895–911, 2010.

[Chen et al., 2015] Guangyong Chen, Fengyuan Zhu, andPheng-Ann Heng. An efficient statistical method for im-age noise level estimation. In ICCV, pages 447–485, 2015.

[Dabov et al., 2007a] K. Dabov, A. Foi, V. Katkovnik, andK. Egiazarian. Color image denoising via sparse 3D col-laborative filter with grouping constraint in Luminance-Chrominance space. In ICIP, pages 313–316, 2007.

[Dabov et al., 2007b] Kostadin Dabov, Alessandro Foi,Vladimir Katkovnik, and Karen O. Egiazarian. Im-age denoising by sparse 3-D transform-domain collabo-rative filtering. IEEE Transactions on Image Processing,16(8):2080–2095, 2007.

[Dong et al., 2013a] Weisheng Dong, Guangming Shi, andXin Li. Nonlocal image restoration with bilateral varianceestimation: A low-rank approach. IEEE Transactions onImage Processing, 22(2):700–711, 2013.

[Dong et al., 2013b] Weisheng Dong, Lei Zhang, Guang-ming Shi, and Xin Li. Nonlocally centralized sparse rep-resentation for image restoration. IEEE Transactions onImage Processing, 22(4):1620–1630, 2013.

[Fang et al., 2020a] Yixiang Fang, Xin Huang, Lu Qin, YingZhang, Wenjie Zhang, Reynold Cheng, and Xuemin Lin.A survey of community search over big graphs. The VLDBJournal, 29:353–392, 2020.

[Fang et al., 2020b] Yixiang Fang, Yixing Yang, WenjieZhang, Xuemin Lin, and Xin Gao. Effective and effi-cient community search over large heterogeneous infor-


643

mation networks. Proceedings of the VLDB Endowment,13(6):854–867, 2020.

[Gu et al., 2017] Shuhang Gu, Qi Xie, Deyu Meng, Wang-meng Zuo, Xiangchu Feng, and Lei Zhang. Weightednuclear norm minimization and its applications to lowlevel vision. International Journal of Computer Vision,121(2):183–208, 2017.

[Hu et al., 2019] Zhentao Hu, Zhiqiang Huang, XinjianHuang, Fulin Luo, and Renzhen Ye. An adaptive nonlocalgaussian prior for hyperspectral image denoising. IEEEGeoscience and Remote Sensing Letters, 16(9):1487–1491, 2019.

[Lebrun et al., 2015] Marc Lebrun, Miguel Colom, andJean-Michel Morel. The noise clinic: A blind image de-noising algorithm. IPOL Journal, 5:1–54, 2015.

[Liu et al., 2017] Weiwei Liu, Ivor W. Tsang, and Klaus-Robert Muller. An easy-to-hard learning paradigm formultiple classes and multiple labels. Journal of MachineLearning Research, 18(94):1–38, 2017.

[Liu et al., 2018] Ding Liu, BihanWen, Xianming Liu,Zhangyang Wang, and Thomas S. Huang. When imagedenoising meets high-level vision tasks: A deep learningapproach. In IJCAI, pages 842–848, 2018.

[Liu et al., 2019] Weiwei Liu, Donna Xu, Ivor W. Tsang, andWenjie Zhang. Metric learning for multi-output tasks.IEEE Transactions on Pattern Analysis and Machine In-telligence, 42(2):408–422, 2019.

[Mairal et al., 2008] Julien Mairal, Michael Elad, andGuillermo Sapiro. Sparse representation for color im-age restoration. IEEE Transactions on Image Processing,17(1):53–69, 2008.

[Mairal et al., 2012] Julien Mairal, Francis R. Bach, andJean Ponce. Task-driven dictionary learning. IEEE Trans-actions on Pattern Analysis and Machine Intelligence,34(4):791–804, 2012.

[Marsousi et al., 2014] Mahdi Marsousi, Kaveh Abhari, PaulBabyn, and Javad Alirezaie. An adaptive approach tolearn overcomplete dictionaries with efficient numbersof elements. IEEE Transactions on Signal Processing,62(12):3272–3283, 2014.

[Nam et al., 2016] Seonghyeon Nam, Youngbae Hwang, Ya-suyuki Matsushita, and Seon Joo Kim. A holistic approachto cross-channel image noise modeling and its applicationto image denoising. In CVPR, pages 1683–1691, 2016.

[Roth and Black, 2005] Stefan Roth and Michael J. Black.Fields of experts: A framework for learning image priors.In CVPR, volume 2, pages 860–867, 2005.

[Starck et al., 2002] Jean-Luc Starck, Emmanuel J. Candes,and David L. Donoho. The curvelet transform for im-age denoising. IEEE Transactions on Image Processing,11(6):670–684, 2002.

[Wang et al., 2018a] Zengmao Wang, Yuhong Guo, andBo Du. Matrix completion with preference ranking fortop-N recommendation. In IJCAI, pages 3585–3591,2018.

[Wang et al., 2018b] Zheng Wang, Xiang Bai, Mang Ye, andShin’ichi Satoh. Incremental deep hidden attribute learn-ing. In ACM MM, pages 72–80, 2018.

[Xie et al., 2016] Yuan Xie, Shuhang Gu, Yan Liu, Wang-meng Zuo, Wensheng Zhang, and Lei Zhang. Weight-ed Schatten p-norm minimization for image denoising andbackground subtraction. IEEE Transactions on Image Pro-cessing, 25(10):4842–4857, 2016.

[Xu et al., 2015] Jun Xu, Lei Zhang, Wangmeng Zuo, DavidZhang, and Xiangchu Feng. Patch group based nonlocalself-similarity prior learning for image denoising. In IC-CV, pages 244–252, 2015.

[Xu et al., 2017a] Chen Xu, Zhouchen Lin, and HongbinZha. A unified convex surrogate for the Schatten-p nor-m. In AAAI, pages 926–932, 2017.

[Xu et al., 2017b] Jun Xu, Lei Zhang, David Zhang, and Xi-angchu Feng. Multi-channel weighted nuclear norm min-imization for real color image denoising. In ICCV, pages1096–1104, 2017.

[Ye et al., 2018] Mang Ye, Zheng Wang, Xiangyuan Lan,and Pong C. Yuen. Visible thermal person re-identificationvia dual-constrained top-ranking. In IJCAI, pages 1092–1099, 2018.

[Zhang and Aeron, 2016] Zemin Zhang and Shuchin Aeron.Denoising and completion of 3D data via multidimension-al dictionary learning. In IJCAI, pages 2371–2377, 2016.

[Zhang et al., 2013] Min Zhang, Zheng-Hai Huang, and Y-ing Zhang. Restricted p-isometry properties of nonconvexmatrix recovery. IEEE Transactions on Information Theo-ry, 59(7):4316–4323, 2013.

[Zhang et al., 2017a] Kai Zhang, Wangmeng Zuo, YunjinChen, Deyu Meng, and Lei Zhang. Beyond a Gaussian de-noiser: Residual learning of deep CNN for image denois-ing. IEEE Transactions on Image Processing, 26(7):3142–3155, 2017.

[Zhang et al., 2017b] Kai Zhang, Wangmeng Zuo, ShuhangGu, and Lei Zhang. Learning deep CNN denoiser prior forimage restoration. In CVPR, pages 2808–2817, 2017.

[Zhang et al., 2018] Kai Zhang, Wangmeng Zuo, and LeiZhang. FFDNet: Toward a fast and flexible solution forCNN-based image denoising. IEEE Transactions on Im-age Processing, 27(9):4608–4622, 2018.

[Zhu et al., 2016] Fengyuan Zhu, Guangyong Chen, andPheng-Ann Heng. From noise modeling to blind imagedenoising. In CVPR, pages 420–429, 2016.


644

Documents

Multichannel Color Image Denoising via Weighted Schatten p ... · challenges faced by RGB color image denoising. Intuitive-ly, if we apply the grayscale image denoising algorithm