Upload
others
View
20
Download
1
Embed Size (px)
Citation preview
Pixelwise View Selection for Unstructured Multi-View StereoJohannes L. Schönberger1 Enliang Zheng2 Marc Pollefeys1,3 Jan-Michael Frahm2
1ETH Zürich 2UNC Chapel Hill 3Microsoft
Overview
Joint Depth - Normal - Occlusion InferenceThis work presents an open source Multi-View Stereo system for robust and efficient dense modeling from unstructured image collections. Experiments on benchmarks and large-scale Internet photo collections demonstrate state-of-the-art performance in terms of accuracy, completeness, and efficiency.
Contributions
• Joint depth - normal - occlusion inferenceembedded in improved PatchMatch sampling scheme
• Pixelwise view selectionusing photometric and geometric priors
• Multi-view geometric consistencyfor simultaneous refinement and image-based fusion
• Graph-based filtering and fusionof depth and normal maps
Multi-View Geometric Consistency
Pixelwise View Selection
• Occlusion prior
• Triangulation prior
• Resolution prior
• Incident prior
Ref. patch Source patches
Reference camera
Source camera
Source camera
-1 -0.5 0 0.5 1
NCC score
0
0.5
1
Occ
lusi
on P
rior
σρ = 0.3
σρ = 0.6
σρ = 0.9
0 50 100 150
Incident angle [deg]
0
0.5
1
Inci
dent
Prio
r
σκ = 15◦
σκ = 30◦
σκ = 45◦
0 5 10 15 20 25 30
Triangulation angle [deg]
0
0.5
1
Tria
ngul
atio
n P
rior
α = 2◦
α = 5◦
α = 10◦
α = 15◦
0 1 2 3 4 5
Relative resolution blm with b
l = 1
0
0.5
1
Res
olut
ion
Prio
r
Source patchesRef. patch
Traditional Pixelwise
a
S
Pix
• Joint likelihood function
• Generalized Expectation Maximization (GEM) •E-Step: Infer using variational inference •M-Step: Infer using PatchMatch sampling
Z θ,N
ξml = 1−ρml +ηmin (ψml , ψmax)
Photometric Geometric
Cost function
Optimization argminθ∗
l,n∗
l
1
|S|
∑m∈S
ξml (θ∗l ,n∗
l )
Filtering and Fusion
ψml
Filtering
Fusion
ResultsZheng et al. Photometric Photometric + Geometric Filtered Normals
https://colmap.github.ioSource Code | Documentation | Tutorial | Examples
P (αml ) = 1−
(min(α,αm
l)−α)2
α2
P (βml ) = min(βm
l , (βml )−1)
P (κml ) = exp(−
κm
l
2
2σ2κ
)
P (Xml |Zm
l , θl) =
{1
NAexp
(−
(1−ρm
l(θl))
2
2σ2ρ
)if Zm
l = 1
1
NU if Zm
l = 0
P (X,Z, θ,N)
NormalsDepthOcclusionImages
P (αml )
P (βml )
P (κml )
P (Xml |Zm
l , θl)