Using multiple optical images 3D surface reconstrunction

Using multiple optical images3D surface reconstrunction

20205309 Chungsu Jang

2020 12.10

●Introduction

●Related Works

•Approach & Experiment

●Conclusion

●Q&A

Contents

DSM(Digital Surface Model)

Korea Daejeon, WorldView3 Satellite

Test area 1

Test area 2

Analysis Data characteristics3D dense points obtained using satellite imagery and SGM(Semi Global Mapping) matching technique

Test area 1

Test area 2

Area 1(Daejeon) : Disparity Maps

Area 2(Daejeon) : Disparity Maps

Refinementblock_size = 15

➢ All Census & MI data : noise vertical range ~ 2 m, Many miss-matching points

Refinement

Problem Derivation through Analysis Data characteristics

●Introduction

●Related Works

•Approach & Experiment

●Conclusion

●Q&A

Contents

Related Works

●Urban 3D reconstruction– Urban Semantic 3D reconstruction from Multiveiw Satellite Imagery, CVP

R 2019

Related Works

• Geo referencing –mapping image pixels to global coordinates, photometry

GtoI()

ItoG()

Ref: Introduction to Photogrammetry and Remote sensing, 1st Edition

R : RPC(The rational polynomial coefficient)

Related Works

• Semi Global Matching(SGM)– Algorithm and architecture of disparity estimation with mini-Census

adaptive support weight, IEEE Transactions on Circuits, 2006

Related Works

• Point cloud set matching– ICP(Iterative Closest Point)

• A Method for Registration of 3-D Shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992

–Least-squares Height Difference(LHD) Matching• Three-dimensional absolute orientation of stereo models using digital elevation models,

PHOTOGRAMMETRIC ENGINEERING AND REMOTE SENSING, 1988

●Introduction

●Related Works

•Approach & Experiment

●Conclusion

●Q&A

Contents

For 3D building shape restoration , approach

High-rise Building Reconstructionusing Warped Images

No way to recover these missing buildings?

Projected points from image # 1

Alignment and color information in the building

Left P.C.

Left image

1 4

Building

Object space

Plane

segmented

from LiDAR

2

3

Right P.C.

Right image

1 4

Building

Object space

Plane

segmented

from LiDAR

2

3

Projected points from image # 2

Warped Image

When knowing information about the floor plan of the building

Alignment and color information in the building

Projected points from image # 1

Left P.C.

Left image

1 4

Building

Object space

Plane

segmented

from LiDAR

2

3

Two warped image comparison /

Inside and outside the building Similarity

Warped Image

fl frelbl dl erbr dr

LiDAR plane

PCL PCR

A

B

C

D E

Ground

Rooftop

Wall

Hig

h s

imila

rity

Lo

w s

imila

rity

Hig

h s

imila

rity

F

Hig

h s

imila

rity

Left warped image

Right warped image

g(bl) g(dl) g(el) g(fl)

g(br) g(dr) g(er) g(fr)

Left satellite image

Right satellite image

Left warped image

Building rooftop plane

Right warped image

➢ Similarity measure between two warped images and building height estimation

High-rise Building Reconstructionusing Warped Images

Similarity Check: Hamming distance

19

Proposed Workflow – 1st track approach

Two warped image generation

Panchromatic Image (Right)

Min < Height< Max

Similarity Calculation (Hamming Dist.)

Best Height Estimation

Panchromatic Image (Left)

DBM generation

Yes

No

Check the min. costRecord costs

for each height

Best Views

Three-dimensional geometric condition

Convergence Angle : 25~45

Bisector Elevation Angle : > 60

Satellite Azimuth Angle : <70

Satellite Azimuth Angle : <15

Similarity Measure Map

Left warped image

Right warped image

High similarity

Low similarity

Similarity Check

➢ High similarity measure at the correct height

➢ Low similarity measure at the incorrect height

H : 81m H : 107mH : 99m

Test results

Best estimated Building Height : 99m at the min. cost

※ Reference height is extracted by intersection after manually measuring the same feature from left and right satellite images

Test results

Reference Height (m)

Estimated Height(m)

diff (m)

‘ㄷ’type building 72.61 72.50 0.11

‘ㅁ’type building 99.29 99.00 0.29

Apartment 100.75 101.00 0.25

High building 233.45 233.50 0.05

Low builidng– 1 70.47 70.00 0.47

Low building– 2 70.71 71.00 0.29

24

Using Warped image Outline and plane extraction method

Warped image (Right)

Line matching

Warped image (Left)

Best height

Line extraction (Left) Line extraction (Right)

Line merge (Left) Line merge (Right)

Line intersection

Space partitioning

Cumulative min cost image generation

Space Selection

Space Union

DBM

Cost check for each partition

Linear Information Extraction and Merge

Line detection Line merge

Line detection Line merge

Left warped image at the Best height

Right warped imageat the Best height

Best height estimation using similarity

measure

Matching, Intersection, Space partitioning and Building area selection

Line matching Line intersection Space Partitioning

Space SelectionSpace Union DBM

Test results (two satellite images, Redundancy)

ㄷtype

ㅁtype

Hig

h

Space partitioning Space selection Space unionImage

Space partitioning Space selection Space unionImage

Space partitioning Space selection Space unionImage

Apartm

ent

Low

-

1

Low

-

2

Space partitioning Space selection Space unionImage

Space partitioning Space selection Space unionImage

Space partitioning Space selection Space unionImage

Test results (two satellite images, Redundancy)

DBM

ㄷtype

ㅁtype

Hig

h

DBM

DBM

Apartm

ent

Low

-1Low

-

2

DBM

DBM

DBM

Test results (two satellite images, Redundancy)

Test results (Improved DSM Generation)

DSM from point cloud

Build

ing 1

Build

ing 2

Build

ing 3

DSM from point cloud

DSM from point cloud

Improved DSM

Improved DSM

Improved DSM

Test results (Improved DSM Generation)

Build

ing 4

Build

ing 5

DSM from point cloud Improved DSM

DSM from point cloud Improved DSM

Sensor Fusion

IKONOS-2

Test area

7340 7340

7341 7341

Epipolar Image SGM Disparity Map

WV-3

Sensor Fusion

7340 7341

DEM

IKONOS-2 WV-3

Sensor Fusion

Point Cloud Matching : Use 12,322 points

RMSE = 23.4m; Max = 89.1m

LHD Matching (reference)

Sensor Fusion

3D matching

(7340+7341)

Point Cloud

Matching

Mosaic

(7340+7341)

●Introduction

●Related Works

•Approach & Experiment

●Conclusion

●Q&A

Contents

Conclusion

➢ Suggest to generate 3d surface model from several sat

ellite image

➢ Show you the generated 3d model surface, visually

➢ Limit : Completeness? / Accuracy?

●Introduction

●Related Works

•Approach & Experiment

●Conclusion

●Q&A

Contents

Thank youQ & A