Embed Size (px)
Citation preview
Unfolding an Indoor Origami World
David Fouhey, Abhinav Gupta, Martial Hebert
1
2
3
Local Evidence
4 Hoiem et al. 2005, Saxena et al. 2005, Fouhey et al. 2013, etc.
Constraints
5
Constraints for Single Image 3D
6
Local Smoothness
Low Level, Generic
Hoiem et al. 2005, Saxena et al. 2005, 2008, Munoz et al., 2009, etc.
Constraints for Single Image 3D
7
Local Smoothness
Low Level, Generic
Hoiem et al. 2005, Saxena et al. 2005, 2008, Munoz et al., 2009, etc.
Constraints for Single Image 3D
8
Low Level, Generic
High Level, Physical
High Level, Physical
Constraints for Single Image 3D
9
Local Smoothness
Low Level, Generic
Coughlan and Yuille 2000, etc.
High Level, Physical
Constraints for Single Image 3D
10
Local Smoothness
Low Level, Generic
Hedau et al. 2009, Del Pero et al., 2011, Wang et al., 2012, Schwing et al. 2012, etc.
High Level, Physical
Constraints for Single Image 3D
11
Local Smoothness
Low Level, Generic
Lee et al. 2010, Xiao et al. 2012, Zhao et al. 2013, Schwing et al., 2013, etc.
High Level, Physical
Constraints for Single Image 3D
12
Low Level, Generic
Mid-level in the Past
13 Huffman 71, Clowes 71, Kanade 80, 81 Sugihara 86, Malik 87, etc.
Our Mid-Level Constraints
14
This Work
15
Input: Single Image
Output: Discrete Scene Parse
Overview
16
Parameterization Formulation
Experimental Results
Overview
17
Parameterization Formulation
Experimental Results
Parameterization
18
Parameterization
19
vp1
vp2
vp3
VP Estimator from Hedau et al., 2009
Parameterization
20
Two VPs give grid cell
Encoding Surface Normals
21
Encoding Surface Normals
22
Encoding Surface Normals
23
Encoding Surface Normals
24
x1,…, x400 x401,…, x800 x801,…, x1200
Related Parameterizations
25
vp1
vp3
x2
x1
x3
x4
Hedau et al., 2009; Wang et al. 2010, Schwing et al., 2012, 2013
vp2
Overview
26
Parameterization Formulation
Experimental Results
Parameterization
27
Formulation
28
Unaries
29
Unaries
30
Low c Any 3D Evidence
High c
Unaries
31
…
Input 3D Primitive Bank
Local: Data-Driven 3D Primitives
Fouhey, Gupta, Hebert, 2013
Unaries
32 Hedau, Hoiem, Forsyth, 2009
Global: Cuboid Fit + Clutter Mask
Input Predicted Walls Clutter Mask
Binaries
33
Convex/Concave Constraints
34
Convex (+) Concave (-)
Convex/Concave Constraints
35
Detected Concave (-)
Convex/Concave Constraints
36
Detected Concave (-)
Convex/Concave Constraints
37
Detected Concave (-)
Convex/Concave Constraints
38
Detected Concave (-)
Convex/Concave Constraints
39
Detected Concave (-)
Detecting Convex/Concave
40
Ground-Truth Discontinuities similar to Gupta, Arbelaez, Malik, 2013 3DP from Fouhey, Gupta, Hebert, 2013
Input 3D Primitive Bank
…
Use 3DP to Transfer Discontinuities
Smoothness
41
Constraints
42
Solving the Model
43
Overview
44
Parameterization Formulation
Experimental Results
Dataset
NYU Depth v2: 795 Train, 654 Test
45
Qualitative Results
46
Qualitative Results
47
Qualitative Results
48
Qualitative Results
49
Surface Connection Graphs
50
Convex Concave
Baseline
Primary Baseline: 3D Primitives
51 Fouhey, Gupta, Hebert, 2013
Output Input
Quantitative Results
52
Summary Stats (⁰) (Lower Better)
% Good Pixels (Higher Better)
11.25⁰ 22.5⁰ 30⁰ Mean Median
3DP 35.9 52.0 57.8 36.0 20.5 49.4
Hedau et al. 34.2 49.3 54.4 40.0 23.5 54.1
RMSE
Lee et al. 18.6 38.6 49.9 43.3 36.3 54.6
Proposed 37.6 53.3 58.9 35.1 19.2 48.7
Quantitative Results
53
Failure Modes
Mistaken but Confident Evidence
54
Failure Modes
55
Missing High-Level Modeling
Conclusions
56
Single Image
Parameterization
Formulation
Discrete Parse
Thank You
57
