Multi-View Reconstruction Preserving Weakly-Supported Surfaces
(CVPR 2011)M. Jancosek and T. Pajdla
Czech Technical University in Prague
Presenter : Jia-Hao Syu
04/21/23 1
Motivation
04/21/23 2
Outline
• Related Work[15]– System diagram
• Weakly-Supported Surfaces• Idea• Modified weights• Results• Conclusion
04/21/23 3
Related Work
• [15] P. Labatut, J. Pons and R.Keriven, “Robust and efficient surface reconstruction from range data”, In Computer Graphics Forum, 2009
• Target : Reconstruct a surface from a set of merged scans (noisy and outliers)04/21/23 4
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization
of a cloud point
3D Delaunay Triangulation
Surface Reconstruction by graph-cut method
04/21/23 5
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization
of a cloud point
3D Delaunay Triangulation
Surface Reconstruction by graph-cut method
04/21/23 6
3D Scanning Technique
• Contact• Non-Contact– Time-of-flight camera : a range imaging camera
system that resolves distance based on the known speed of light
D : distance c : speed of lightt : time for round-trip between A and B
04/21/23 7
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization
of a cloud point
3D Delaunay Triangulation
Surface Reconstruction by graph-cut method
04/21/23 8
3D Cloud Points
• Acquire a depth map of each camera by 3D scanning technique
• Compute depth maps of a 3D cloud points to every camera by plane-sweeping method
04/21/23 9
• Pick one pixel P with depth d
Plane-sweeping Method
P
d
Reference Image
04/21/23 10
• Find the nearest n target cameras(ex. n = 4)
Plane-sweeping Method
P
d
Reference Image
04/21/23 11
• Compute photo consistency by normalized cross-correlation(NCC)
Plane-sweeping Method
P
d
Reference Image
04/21/23 12
Target Image
Plane-sweeping Method
• Normalized Cross-Correlation
04/21/23 13
The formulation is from wikin : the pixel numberf(x,y) : reference imaget(x,y) : target image
The value of NCC is between -1 and 1
One 3D Cloud Points
• Compute photo-consistence between reference and target image
• One 3D cloud points can be built
• Get all depth-maps of each camera by using a related camera matrix
04/21/23 14
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization
of a cloud point
3D Delaunay Triangulation
Surface Reconstruction by graph-cut method
04/21/23 15
2D Delaunay Triangulation
• Three points can draw a triangle
• Add one more point
04/21/23 16
or
2D Delaunay Triangulation
• Draw a circumcircle of triangle
04/21/23 17
or
• Give a set of P point in 2D
2D Delaunay Triangulation
04/21/23 18
2D Delaunay Triangulation
• No point in P is inside the circumcircle of any triangle
3D Delaunay triangulation : no point in P is inside the circumsphere of any tetrahedralization
04/21/23 19
3D Delaunay triangulation
• Example for 3D Delaunay triangulation
04/21/23 20
System Diagram
3D cloud points for each cameras
Combine to one 3D cloud points
Delaunay tetrahedralization
of a cloud point
3D Delaunay Triangulation
Surface Reconstruction by graph-cut method
04/21/23 21
Surface Reconstruction
• We build the 3D Delaunay triangulation
• How do you reconstruct surface of the object?
• Concept : 3D cloud points are dense near the object surface (cost is small)
• S-t graph cut algorithm
04/21/23 22
04/21/23
• Source and sink become separated the node of set by a cut
• the cost of a cut :
• Minimum cut : a cut whose cost is the least over all cuts
S-t Graph Cut
23
Define Parameters
• Node : Delaunay tetrahedralization• Edge : triangulation between adjacent
tetrahedralizations • s(source) : outside of the surface• t(sink) : inside of the surface
04/21/23 24
P
S-t Graph Cut Algorithm
• Perform a Delaunay Triangulation of the 3d point cloud
04/21/23 25
S-t Graph Cut Algorithm
• Add a node P from left tetrahedralization
04/21/23 26
P
S-t Graph Cut Algorithm
• Add a node Q from right tetrahedralization
04/21/23 27
P Q
S-t Graph Cut Algorithm
• Add two s and t nodes
04/21/23 28
P Q
s
t
S-t Graph Cut Algorithm
04/21/23 29
P Q
s
t
10
100
0
3
S-t Graph Cut Algorithm
04/21/23 30
Outside the surface
inside the surface
This is the surface we want
Assigned Weight
• 3D cloud points to camera center(line of sight)
04/21/23 31
04/21/23 32
Formulation of Cost Function
: Visibility Information from points, cameras
: Quality of reconstructed surface in terms of size of triangles
04/21/23 33
Weakly Supported Surfaces
Not photo consistent surface : Low-textured walls, windows, cars and ground planes
Idea
• Other information to reconstruct weakly supported surface
• Visual Hull
04/21/23 35
Idea
• Define free-support-space
• Highly-supported-free space : union of dense 3D points
• Weakly-supported surface with weakly sampled by 3D points are close to the highly-supported-free space
04/21/23 36
Free-space-support
04/21/23 37
pi
pj
T
r
Original 3D cloud points
X : 3D cloud points(before photo-consistence)
Target
• Large Jump in Free Space Support as we go from outside to inside.
• Next, I give a example of weight assumption
Old T-weights
Modified Weights
Setting up t-weight
04/21/23 41
System and Spend Time
DataSet/Method Baseline[CFG 09](mins) Ours(mins)
Castle 30 32
Dragon 90 94
• System OS : 64-bit Win7 CPU : Inter Core i7 RAM : 12GB
• Dataset Castle data : 30 images with 3072*2048 resolution Dragon data : 114 images with 1936*1296 resolution
Results
INPUT IMAGE POINT CLOUD
CFG-09 OUR METHOD
Results
INPUT IMAGE POINT CLOUD
CGF-09 OUR METHOD
Demo Video
• Images :http://www.youtube.com/watch?
v=uwluzq5LUn0&feature=player_embedded• Demo videohttp://www.youtube.com/watch?v=UgkB7ITpNaE&feature=player_embedded
04/21/23 45
Conclusion
• Resolve weakly-supported surface by using the information of free-support-space
04/21/23 46
Reference
• M. Jancosek and T. Pajdla, “Multi-View Reconstruction Preserving Weakly-Supported Surface”, IEEE Conference on Computer Vision and Pattern Recognition, 2011
• P. Labatut, J. Pons and R.Keriven, “Robust and efficient surface reconstruction from range data”, In Computer Graphics Forum, 2009
• P. Labatut, J. Pons, and R. Keriven., “Efficient Multi-view Reconstruction of Large-Scale Scenes using Interest Points, Delaunay Triangulation and Graph Cuts”, International Conference on Computer Vision, 2007
04/21/23 47
Reference
• M. Jancosek and T. Pajdla , ”Hallucination-free multi-view stereo.”, In RMLE , 2010
• M. Jancosek and T. Pajdla, “Removing hallucinations from 3D reconstructions”, Technical Report CMP CTU, 2011
04/21/23 48