89
Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt

Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

  • Upload
    others

  • View
    11

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Representation, Tracking, and

MappingTanner Schmidt

Page 2: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- SDF limitations- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion

Page 3: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- SDF limitations- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion

Page 4: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Explicit Surface Representations

- Geometry is stored explicitly as a list of points, triangles, or other geometric fragments- e.g. meshes, point clouds

Vertices: [ (x0, y0, z0), (x1, y1, z1), …, (xn, yn, zn) ]

Indices: [ (i0, i1), (i2, i3), …, (in-1, in) ]

Page 5: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

- Geometry is not stored explicitly but rather defined as a level set of a function defined over the space in which the geometry is embedded

- There are parametric representations:

Implicit Surface Representation

Page 6: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

- Geometry is not stored explicitly but rather defined as a level set of a function defined over the space in which the geometry is embedded

- And there are nonparametric representations:

Implicit Surface Representation

Page 7: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

- Geometry is not stored explicitly but rather defined as a level set of a function defined over the space in which the geometry is embedded

- And there are nonparametric representations:

Implicit Surface Representation

Page 8: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Implicit to Explicit Conversion

- In two dimensions, we can use an algorithm called marching squares

Page 9: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Implicit to Explicit Conversion in 3D

- Typically done using marching cubes, a 3D analogue to marching squares

Page 10: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Implicit to Explicit Conversion in 3D

- Can also be done by raycasting for a view-dependent partial surface

Page 11: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

- Can be done by finding the closest point between each discrete location and any part of the geometry

Explicit to Implicit Conversion

Page 12: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

- Can also be done with a distance transform

Explicit to Implicit Conversion What if we want to build surface representations from raw (noisy) observations?

Page 13: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- SDF limitations- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion

Page 14: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 15: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 16: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

1.356 1.384 1.422 1.511 1.607

Page 17: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

1.356 1.384 1.422 1.511 1.607

Page 18: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 19: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 20: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 21: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 22: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 23: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 24: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 25: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 26: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 27: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 28: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 29: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 30: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 31: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 32: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 33: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 34: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 35: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 36: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 37: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 38: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 39: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 40: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 41: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 42: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 43: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 44: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 45: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 46: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 47: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Fusion

Page 48: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

- This addition requires the per-frame projected truncated signed distance volumes to be globally registered

Signed Distance Function Fusion

Page 49: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- SDF limitations- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion

Page 50: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Tracking

Page 51: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Tracking

Page 52: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Tracking

Page 53: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Tracking

Page 54: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Tracking

Page 55: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Tracking

Page 56: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Tracking

Page 57: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Signed Distance Function Tracking

Page 58: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Point-plane Iterative Closest Point (ICP)

Page 59: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Point-plane Iterative Closest Point (ICP)

Page 60: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Point-plane Iterative Closest Point (ICP)

Page 61: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Point-plane Iterative Closest Point (ICP)

Page 62: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Point-plane Iterative Closest Point (ICP)

Page 63: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Point-plane Iterative Closest Point (ICP)

Page 64: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Point-plane Iterative Closest Point (ICP)

Page 65: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Point-plane Iterative Closest Point (ICP)

Page 66: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Point-plane Iterative Closest Point (ICP)

Page 67: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Direct Signed Distance Function Tracking

Page 68: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Direct Signed Distance Function Tracking

Page 69: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Direct Signed Distance Function Tracking

Page 70: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Online fusion

- Tracking requires the fused SDF volume for all frames up to the current frame

Page 71: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Online fusion

- Tracking requires the fused SDF volume for all frames up to the current frame- We must maintain a running average SDF value at each cell

Page 72: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Online fusion

- Tracking requires the fused SDF volume for all frames up to the current frame- We must maintain a running average SDF value at each cell- Each cell stores both an SDF value and a weight

Page 73: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Truncated Signed Distance Function

Page 74: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- SDF limitations- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion

Page 75: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

● Size: ○ In general, scales linearly with the volume of the computed distance field, and exponentially in the

resolution● Thin surface representation:

○ If the resolution is insufficient, observations from opposing sides of thin surfaces can “cancel out”

SDF Limitations

Page 76: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

● Size: ○ In general, scales linearly with the volume of the computed distance field, and exponentially in the

resolution● Thin surface representation:

○ If the resolution is insufficient, observations from opposing sides of thin surfaces can “cancel out”

SDF Limitations

Page 77: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- SDF limitations- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion

Page 78: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

KinectFusion [Newcombe et al.]

Page 80: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion

Page 82: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion

Page 84: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion

Page 88: Representation, Tracking, and Signed Distance …...Signed Distance Function Representation, Tracking, and Mapping Tanner Schmidt Overview - Explicit and implicit surface representations

Overview

- Explicit and implicit surface representations- SDF fusion- SDF tracking- Related research

- KinectFusion- KinTinuous- BundleFusion- DART- DynamicFusion