Measuring Depth and Velocity withDefocus and Differential Motion

Emma Alexander1, Qi Guo1, Sanjeev Koppal2, Steven Gortler1, Todd Zickler1 1 Harvard SEAS2 University of Florida

Focal Flow

Motivation Low power (mW) depth sensing

[Rubenstein et al. 14] [Ma et al. 13]

~200 mW ~20 mW

Contribution

Optical Flow

Depth & 3D Velocity

Focal Flow

Image Motion

[Photo: Tony Hisgett]

Idea Combine motion and defocus blur

Wid

e ap

ertu

reP

inho

le

[Photo: Lost and Taken]

Textured plane

Pinhole

Wide aperture(Thin-lens Model)

In-focus plane

Optical Flow

Residual

Derivation Gaussian blur reveals depth

=Z

Z − Zf(2k + rkr) ∗ P = v(Z, �X) m ∗ k ∗ P

Theorem The residual can be factored into scene information and an image convolution exactly when the blur is Gaussian and the operator is the Laplacian, i.e.

m ∗ k ∝ 2k + rkrm ∗ k ∝ 2k + rkrm ∗ k ∝ 2k + rkrm ∗ k ∝ 2k + rkrm ∗ k ∝ 2k + rkrm ∗ k ∝ 2k + rkrm ∗ k ∝ 2k + rkrm ∗ k ∝ 2k + rkr

m ∗ k ∝ 2k + rkrm ∗ k ∝ 2k + rkrm ∗ k ∝ 2k + rkr? ×

Filter

R(Z, �X, filter k, pinhole image P ) ∝ m ∗ I

m ∗ k ∝ 2k + rkr

wmk = −rkr

k ∝ e−w

∫r

0m(s)s

ds

m ∝ rn

n ∈ {2, 4, 6, ...}

n = 2

Texture independence

Fourier transform

Solve differential equation

Compact operator

Nonnegative transmittance

All kernels from same filter

Inverse Fourier Transform

True depth (mm)250 350 450 550 650 750

Est

imat

ed d

epth

(m

m)

250

350

450

550

650

750

Proof of Concept

x

yTrue

dep

th (

mm

)

x

y

Est

imat

ed d

epth

(m

m)

x

y

Est

imat

ed d

epth

(m

m)

x

y

10mm

10mm

x

y

x

y

Input Image True Depth Result Sample PSF

Experimental results

[Photo: Thorlabs]True depth (mm)

Est

imat

ed s

peed

(m

m/fr

ame)

250 350 450 550 650 7500

0.2

0.4

0.6

0.8

1

1.2

Code, equipment, results: https://vision.seas.harvard.edu/focalflow