Upload
aaron-karper
View
156
Download
5
Tags:
Embed Size (px)
Citation preview
Bayesian Depth-from-Defocus with Shading Constraints
Aaron Karperpaper by
Chen Lin Shuochen Su Yasuyuki Matsushita Kun Zhou Stephen Lin
Dec 17th 2013paper: 2013
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 1 / 16
1 Overview of Depth from DefocusMAP estimate
2 Overview of Depth from Shading
3 Optimizing both ModelsDepth-from-defocus
4 Results
5 Discussion
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 2 / 16
Overview of Depth from Defocus
Overview of Depth from Defocus
0 = F−1 − d−1 − v−1d In focus
b =Rv
2
∣∣F−1 − d−1 − v−1d
∣∣ spread out of focus
Model Point-spread as Gaussian
blur h(p|σ2) ∝ exT x2σ2
σ = γb → calibrate γ
I2(p) = I1(p) ∗ h(p|σ22 − σ2
1︸ ︷︷ ︸basically d
)
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 3 / 16
Overview of Depth from Defocus MAP estimate
Overview of Depth from Defocus – MAP estimate
d̂ = arg maxd
P(d|I1, I2)
= arg maxd
P(I1, I2|d)︸ ︷︷ ︸N around deconv
P(d)︸︷︷︸N (c, 1
2λ )
= arg mind
L(I1, I2|d) + L(d)
=∑
p∈pixels
(I1(p) ∗ h(p|d)− I2(p))2
λ∑i,j∈ε
(di − dj)2
Standard Depth from Defocus
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 4 / 16
Overview of Depth from Defocus MAP estimate
Overview of Depth from Defocus – MAP estimate
d̂ = arg maxd
P(d|I1, I2)
= arg maxd
P(I1, I2|d)︸ ︷︷ ︸N around deconv
P(d)︸︷︷︸N (c, 1
2λ )
= arg mind
L(I1, I2|d) + L(d)
=∑
p∈pixels
(I1(p) ∗ h(p|d)− I2(p))2
λ∑i,j∈ε
(di − dj)2
Standard Depth from Defocus
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 4 / 16
Overview of Depth from Defocus MAP estimate
Overview of Depth from Defocus – MAP estimate
d̂ = arg maxd
P(d|I1, I2)
= arg maxd
P(I1, I2|d)︸ ︷︷ ︸N around deconv
P(d)︸︷︷︸N (c, 1
2λ )
= arg mind
L(I1, I2|d) + L(d)
=∑
p∈pixels
(I1(p) ∗ h(p|d)− I2(p))2
λ∑i,j∈ε
(di − dj)2
Standard Depth from Defocus
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 4 / 16
Overview of Depth from Defocus
Overview of Depth from Defocus
Advantages:
Passive perception
Single camera
Advances in lense technology
Precise
Disadvantages:
Lense aperture necessary
Needs texture
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 5 / 16
Overview of Depth from Shading
Overview of Depth from Shading
s(n) = 〈Vn,n〉M
n: normals
M: illumination
Measure M with Lambertian sphere.
Uniform albedo:
L(d) = λ∑i,j∈ε
⟨(pj − pi ),
ni + nj‖ni + nj‖
⟩If albedo present, remove with depth estimate.
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 6 / 16
Overview of Depth from Shading
Overview of Depth from Shading
Advantages:
Passive perception
Single camera
Very precise if applicable
Disadvantages:
Texture hinders perception
Calibration necessary
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 7 / 16
Optimizing both Models Depth-from-defocus
Optimizing both Models – Depth-from-defocus
Cyclic dependencies in L(I1, I|d)
Resolved by magic1
1Markov random fields and loopy belief propagation. Explanation if requested: 15Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 8 / 16
Optimizing both Models
Optimizing both Models
Depth estimate depends on depth-from-defocus, depth-from-shading.
Depth-from-shading profits from depth estimate for albedo removal.
therefore estimation-maximization
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 9 / 16
Results
Results
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 10 / 16
Results
Results
Better results than smoothnessprior.
Especially on untextured regions.
As good results even on texturedregions.
Calibration necessary.
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 11 / 16
Discussion
Discussion
Calibration hinders application.
Not clear why magic2 wasn’t extended to integrate depth from shading,depth from defocus.
Not clear why sensor fusion wasn’t done.
2Markov random fields and loopy belief propagation. Explanation if requested: 15Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 12 / 16
Discussion
Questions?
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 13 / 16
Discussion
Sensor Fusion
d
depth
I
intensity
S1 = N (d , σ1(I ))
S2 = N (d , σ2(I ))
Estimate global σ for both models.
Use to estimate d for each pixel.
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 14 / 16
Discussion
Markov random fields
Each node represents a value (a proposition).
Each node has a belief3.
Each node can depend on other nodes.
Connections if dependency.
Basically undirected Bayes Net.
Solve with Belief Propagation.
3Like a probability, but can take values in R≥0Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 15 / 16
Discussion
Belief Propagation
Nodes send own belief to nodes that depend on them.
Update belief on message.
Pray for convergence.
Aaron Karper paper by Chen Lin, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen LinBayesian Depth-from-Defocus with Shading Constraints Dec 17th 2013paper: 2013 16 / 16