Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Hauptseminar BildanalyseTopic: Single-Image Noise Level Estimation for Blind Denoising
Michael Jobst
June 8, 2016
1/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
The paperSingle-Image Noise Level Estimation for Blind Denoising
Tokyo Institute of Technologyin IEEE Transactions on Image Processing, Vol. 22, No.12,
December 2013
Xinhao Liu Masayuki Tanaka Masatoshi Okutomi
2/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Questions
Single-Image Noise Level Estimation for Blind Denoising
▶ Single-Image:▶ Why?▶ Problems?
▶ Noise▶ What is noise (in the paper)?
▶ Level Estimation▶ What is the noise level?
▶ Blind Denoising▶ Blind vs. non-blind▶ Why?
▶ How to achieve the desired result?
3/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Questions
Single-Image Noise Level Estimation for Blind Denoising
▶ Single-Image:▶ Why?▶ Problems?
▶ Noise▶ What is noise (in the paper)?
▶ Level Estimation▶ What is the noise level?
▶ Blind Denoising▶ Blind vs. non-blind▶ Why?
▶ How to achieve the desired result?
3/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Questions
Single-Image Noise Level Estimation for Blind Denoising
▶ Single-Image:▶ Why?▶ Problems?
▶ Noise▶ What is noise (in the paper)?
▶ Level Estimation▶ What is the noise level?
▶ Blind Denoising▶ Blind vs. non-blind▶ Why?
▶ How to achieve the desired result?
3/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Questions
Single-Image Noise Level Estimation for Blind Denoising
▶ Single-Image:▶ Why?▶ Problems?
▶ Noise▶ What is noise (in the paper)?
▶ Level Estimation▶ What is the noise level?
▶ Blind Denoising▶ Blind vs. non-blind▶ Why?
▶ How to achieve the desired result?
3/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Questions
Single-Image Noise Level Estimation for Blind Denoising
▶ Single-Image:▶ Why?▶ Problems?
▶ Noise▶ What is noise (in the paper)?
▶ Level Estimation▶ What is the noise level?
▶ Blind Denoising▶ Blind vs. non-blind▶ Why?
▶ How to achieve the desired result?
3/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Questions
Single-Image Noise Level Estimation for Blind Denoising
▶ Single-Image:▶ Why?▶ Problems?
▶ Noise▶ What is noise (in the paper)?
▶ Level Estimation▶ What is the noise level?
▶ Blind Denoising▶ Blind vs. non-blind▶ Why?
▶ How to achieve the desired result?
3/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
BM3D denoising (Dabov et al., 2006)
4/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
BM3D denoising (Dabov et al., 2006)
4/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Data representation
▶ input: grayscale image▶ M square patches of 49 pixels▶ image zi , i ∈ M: ground truth (noise free) image patches▶ image yi , i ∈ M: noisy image patches▶ gaussian noise vector ni , variance σ2n, zero-mean▶ yi = zi + ni
5/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Principal Component Analysis▶ Covariance Matrix
Σy =1M
M∑i=1
yiyTi
Comparision: principal components of an image (a) and noise (b)
6/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Principal Component Analysis▶ noise even in all dimensions and added to the image▶ idea: image data does not span the whole space▶ eigenvector of covariance matrix with minimum eigenvalue
represents noise levelλmin(Σy ) = λmin(Σz) + σ2n ∧ λmin(Σz) = 0 ⇔ σ2n = λmin(Σy )
7/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Principal Component Analysis▶ noise even in all dimensions and added to the image▶ idea: image data does not span the whole space▶ eigenvector of covariance matrix with minimum eigenvalue
represents noise levelλmin(Σy ) = λmin(Σz) + σ2n ∧ λmin(Σz) = 0 ⇔ σ2n = λmin(Σy )
7/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Principal Component Analysis▶ noise even in all dimensions and added to the image▶ idea: image data does not span the whole space▶ eigenvector of covariance matrix with minimum eigenvalue
represents noise levelλmin(Σy ) = λmin(Σz) + σ2n ∧ λmin(Σz) = 0 ⇔ σ2n = λmin(Σy )
7/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Principal Component Analysis▶ noise even in all dimensions and added to the image▶ idea: image data does not span the whole space▶ eigenvector of covariance matrix with minimum eigenvalue
represents noise levelλmin(Σy ) = λmin(Σz) + σ2n ∧ λmin(Σz) = 0 ⇔ σ2n = λmin(Σy )
7/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Problem solved
8/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Problem solved?
8/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Patch energy
▶ gradient matrix of patch yi
Gyi =[Dhyi Dv yi
]▶ joint gradient matrix of patch yi
Cyi =GTyi Gyi
▶ trace xii (= sum of eigenvalues) of joint gradient matrix Cyi isthe patch energy
ξi = tr(Cyi )
9/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Patch energy
▶ trace xii (= sum of eigenvalues) of joint gradient matrix Cyi isthe patch energy
ξi = tr(Cyi )
9/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Threshold for patch selection▶ flat patch zf , added noise: yf = zf + n▶ gradient matrix Gyf only depends on the noise n, thus the
noise parameter σ2n▶ possible to derive texture strength of the bare noise ξ(n)▶ approximate ξ(n) by a gamma distribution▶ threshold τ is the 99.9999-th percentile (→ inverse Gamma
cumulative distribution)
10/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Threshold for patch selection
τ = F −1(
δ,N22 ,
2N2 σ
2ntr(DTh Dh + DTv Dv )
)
▶ Problem: we use (the true) σ2n which we do not know▶ chicken-and-egg problem cries for an iterative approach
11/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Threshold for patch selection
τ = F −1(
δ,N22 ,
2N2 σ
2ntr(DTh Dh + DTv Dv )
)
▶ Problem: we use (the true) σ2n which we do not know▶ chicken-and-egg problem cries for an iterative approach
11/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Iterative threshold calculation
12/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Tests
13/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Tests
14/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Tests
15/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Regression model for optimal noise parameter
▶ regression model with unknown parameter vector θ
σ̂′n = R(σ̂0n, σ̂n; θ)
▶ quadratic regression model
σ̂′n = a0 + a1σ̂n + a2σ̂0n + a3σ̂nσ̂0n + a4(σ̂0n)2 + a5(σ̂n)2 + ε
▶ generation of samples by brute force optimization▶ solve with least-squares
16/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Experimental results
17/20 Michael Jobst Hauptseminar Bildanalyse
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
References I
Liu, X., M. Tanaka, and M. Okutomi (2013). “Single-Image NoiseLevel Estimation for Blind Denoising”. In: IEEE Transactions onImage Processing 22.12, pp. 5226–5237. issn: 1057-7149. doi:10.1109/TIP.2013.2283400.
Zhu, Xiang and Peyman Milanfar (2010). “Automatic parameterselection for denoising algorithms using a no-reference measureof image content”. In: Image Processing, IEEE Transactions on19.12, pp. 3116–3132.
Feng, XiaoGuang and Peyman Milanfar (2002). “Multiscaleprincipal components analysis for image local orientationestimation”. In: Signals, Systems and Computers, 2002.Conference Record of the Thirty-Sixth Asilomar Conference on.Vol. 1. IEEE, pp. 478–482.
18/20 Michael Jobst Hauptseminar Bildanalyse
http://dx.doi.org/10.1109/TIP.2013.2283400
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
References II
Dabov, Kostadin et al. (2006). “Image denoising withblock-matching and 3D filtering”. In: Electronic Imaging 2006.International Society for Optics and Photonics,pp. 606414–606414.
Burger, H. C., C. J. Schuler, and S. Harmeling (2012). “Imagedenoising: Can plain neural networks compete with BM3D?” In:Computer Vision and Pattern Recognition (CVPR), 2012 IEEEConference on, pp. 2392–2399. doi:10.1109/CVPR.2012.6247952.
19/20 Michael Jobst Hauptseminar Bildanalyse
http://dx.doi.org/10.1109/CVPR.2012.6247952
Introduction Noise level estimation based on PCA Patch selection Optimal noise parameter Results References
Thank you for your attention!Questions?
20/20 Michael Jobst Hauptseminar Bildanalyse
Gradient covariance matrix▶ gradient at point k = (xk , yk) (Zhu and Milanfar, 2010):
∇f (k) = ∇f (xk , yk) =[
δf (xk , yk)δx ,
δf (xk , yk)δy
]T=: [px (k), py (k)]T
▶ gradient matrix at point k (Feng and Milanfar, 2002):
Gwi =
...
∇f (k)T...
, k ∈ wi▶ joint gradient matrix:
Cwi = GTwi Gwi =[ ∑
k∈wi p2x (k)
∑k∈wi px (k)py (k)∑
k∈wi px (k)py (k)∑
k∈wi p2y (k)
]
Appendix I Michael Jobst Hauptseminar Bildanalyse
Threshold from Gamma distribution
▶ gamma distribution of ξ over the noise free patches:
ξ(n) ∼ Gamma(
N22 ,
2N2 σ
2ntr(DTh Dh + DTv Dv )
)
▶ confidence interval of δ = 1 − 10−6, threshold τ
P(0 < ξ(n) < τ) = δ
Appendix II Michael Jobst Hauptseminar Bildanalyse
IntroductionNoise level estimation based on PCAPatch selectionOptimal noise parameterResultsAppendix