Multiple Light Source Optical Flow Multiple Light Source Optical Flow Robert J. Woodham ICCV’90

Multiple Light Source Multiple Light Source

Optical FlowOptical Flow

Multiple Light Source Multiple Light Source

Optical FlowOptical Flow

Robert J. Woodham

ICCV’90

Introduction

Optical Flow Definition

Is a vector field that shows the direction and magnitude of the intensity changes from one image to the other

Is a vector field that shows the direction and magnitude of the intensity changes from one image to the other

Main Idea

Use the intensity color values recorded from multiple Images of moving objects acquired simultaneously

under different illumination conditions to calculate optical flow

Some considerations

Object Motion vs. brightness change

Not purely geometricDepends on radiometric factors

(illumination, reflectance)

Not purely geometricDepends on radiometric factors

(illumination, reflectance)

The idea is based on. . .

Photometric stereo uses multiple conditions of illumination to determine shape from shading

Theory

Optical Flow Constraint Equation

dE/dt=Exu + Eyv + Et

where

E = E(x,y,t) be the image brightness at point (x,y) as a function of time t

Ex = E/x, Ey = E/y, Et = E/t (partial derivatives of E with respect to x, y and t)

u =dx/dt and v= dy/dt (instantaneous flow in the point (x,y).

Theory (2)

If the brightness does not change as consequence of motion . . .

Exu + Eyv + Et = 0

Validity conditions

Purely translational motion,Orthographic projection

Uniform incident illumination

Theory (3)

Equation properties

•Cannot be solved locally – 1 equation with 2 unknowns

•Variation in scene illumination cause dE/dt0

•Objects acts as indirect sources of illumination (inter-reflection)

•Locations of brightness discontinuity – undefined points.

Exu + Eyv + Et = 0

Using Multiple Light Sources

t

t

yx

yx

E

E

EE

EE

v

u

2

1

1

22

11

E1xu + E1yv + E1t = 0E1xu + E1yv + E1t = 0

E2xu + E2yv + E2t = 0E2xu + E2yv + E2t = 0

::

For 2 light sources

3 Light Sources

E1xu + E1yv + E1t = 0E1xu + E1yv + E1t = 0

E2xu + E2yv + E2t = 0E2xu + E2yv + E2t = 0

E3xu + E3yv + E3t = 0E3xu + E3yv + E3t = 0

Ax = b

x = [u,v]T

b = -[E1t,, E2t ,E3t]T

yx

yx

yx

EE

EE

EE

A

33

22

11

x = (ATA)-1ATbStandard Least Square solution

Overdetermined Problem

Implementation

3 under different illumination condition at time t0

3 same illumination as time t0, with same background but different object position

6 images

3 images taken under different illumination condition in t0

Implementation (2)

Multiple light source optical flow computation at one point

u

v

3 Flow constraint lines

Results

Optical Flow vectors

•Estimation is good where the surface is smoothly shaded

•In the collar dark points degenerate the results

•In the discontinuities, due change of image brightness the estimates is also inaccurate

•Vector in the background due the shadows and inter-reflection

Practical Implementation

Can be used 3 light sources (red, green and blue) continuously illuminating a workspace

The capture can be made using cameras to capture different spectral channels

Conclusion

•The method works better in smoothly curves (not distinct surface markings and the local brightness depends on local shading)

•Restrictions in surface discontinuities and surface markings because local brightness change is dominated by scene features (largely independent of the illumination direction)