Yasutaka Furukawa Satoshi Ikehata Hang Yan Washington University
in St. Louis
STRUCTURED INDOOR MODELING
Room segmentation
Scene
Room Room
Wall Wall
Wall Wall
Wall Wall
Wall Wall
Floor
Ceil
Floor
Ceil
Scene
Room Room
Wall Wall
Wall Wall
Wall Wall
Wall Wall
Floor
Ceil
Floor
Ceil
Door
Room
Wall Wall
Wall Wall
Door
Ceil Floor
Scene
Room Room
Wall Wall
Wall Wall
Wall Wall
Wall Wall
Floor
Ceil
Floor
Ceil Object
Door
Room
Wall Wall
Wall Wall
Detail
Detail
Door
Object
Ceil Floor
Detail
Structure
graph
Structure
grammar
RESULTS
1
0
Scene
Room Room
SUMMARY OUR METHOD
Room reconstruction
Room merging Door addition
Wall/ceiling detail
reconstruction
1
2
3
4
(a) Target (b) Initial (c) Low-rank (d) Low-rank
+ Low-entropy
+ Room reconstruction
+ Door addition
+ Room addition
+ Room merging
+ Detail reconstruction
+ Object reconstruction
Computational time (sec)
Dataset Initialization Room
Segmentation
Room, Floor,
Ceil Recon.
Detail
Recon.
Model
Compilation
Apart. 1 62 86 682 1034 0.5
Apart. 2 48 68 468 558 0.8
Office 1 109 255 1430 1068 1.3
Num. elements Total dist. Error (mm)
Dataset # panos # points Room Wall Obj. Door Wall +Detail
+Object
Apart. 1 16 4933172 6 62 15 5 139 103 81
Apart. 2 15 10065236 5 63 24 4 181 122 86
Office 1 33 4227235 9 113 19 8 155 145 46
Apartment1 Apartment2 Office 1 start/end line
REFERENCES
APPLICATIONS
Tunable reconstruction Inverse CAD
Fig: Automatically generated floorplans
Poisson [2] Vgcuts [3] Target scene Ours
Table: Statistics of reconstructed structured model
Fig: Comparison against other algorithms
Poisson [2] Ours
core free-space
evidence
(a) free-space evidence
(input)
(b) k-medoids
clustering (c) final result
- pick a room node
- walls by shortest-path
based algorithm [1]
- floor, ceiling by RANSAC-
based plane fitting
- pick a pair of wall nodes
- compute point/free-space evidences on walls
- run connection type analysis
- add a door if “door” connection
- merge rooms if “merge” connection
- pick a wall node
- initialize the axis-aligned offset-map
- refine it by enforcing low-entropy (for
less variation of offsets) and low-rank
(for axis-aligned boundaries)
- solve Eq.1 + Eq.2 iteratively
min𝐸,𝐷𝑟
𝐸 1 + 𝜇1 𝐷𝑟 ∗ 𝑠. 𝑡. 𝐷𝑚 = 𝐷𝑟 + 𝐸
min𝐷𝑚
𝐷𝑚 − 𝐷𝑟 22 + 𝜇2 𝛻𝐷𝑚 1 + 𝜇3𝐿𝑎𝑏𝑒𝑙(𝐷𝑚)
𝐷𝑟 𝐷𝑚 𝐷𝑚0
Given 𝐷𝑚, decompose it to low-rank 𝐷𝑟 and sparse 𝐸 (RPCA)
Given 𝐷𝑟 , enforce low-entropy to get 𝐷𝑚 (Label-cost MRF )
Fig: Wall details as 2-D offset-map
Input
[1] R. Cabral and Y. Furukawa. Piecewise planar and compact
floorplan reconstruction from images. CVPR, 2014.
[2] M. Kazhadan and H. Hoppe. Screened poisson surface
reconstruction. SIGGRAPH, 2013.
[3] Y. Furukawa, B. Curless, S. M. Seitz, and R. Szeliski.
Reconstructing building interiors from images. ICCV, 2009.
Problems?
- Existing indoor 3-D models have
no semantics constituting indoor
scenes such as rooms and walls
Contributions? - New reconstruction framework
called structured reconstruction
Manifold mesh
compilation from leaf nodes
Fig: example of the structure graph
Fig: example of the input panorama RGB-D
- Propose structure grammar to
recover a structure graph from
panorama RGB-D images
- Human editable, annotated and
segmented indoor 3-D model
- Various applications such as
indoor scene viewer, inverse
CAD and tunable reconstruction
Assumptions
- Single story building (w/o stairs)
- Room structure (i.e., floor, ceiling,
and walls) is aligned with the
Manhattan directions
Start from the segmentation of 2D
free-space evidence
- Existing indoor 3-D models are
a polygonal soup that is hard to
edit afterwards
(Eq.1)
(Eq.2)
point evidence
optim
ized w
all path
This research is supported by National Science Foundation under
grant IIS 1540012. We thank Floored Inc. for providing us with the
dataset and support.
- run free-space segmentation
- add rooms at cluster centers
Fig: Wall as 1-D polygon
Fig: Connection classification table
