Multispectral Image Invariant to Illumination Colour, Strength, and Shading

Mark S. Drew and Amin Yazdani Salekdeh

School of Computing Science,Simon Fraser University,Vancouver, BC, Canada

{mark/ayazdani}@cs.sfu.ca

Multispectral Image Invariant to Illumination Colour, Strength, and

Shading

Table of ContentsIntroductionRGB Illumination InvariantMultispectral Image FormationSynthetic Multispectral ImagesMeasured Multispectral ImagesConclusion

2

IntroductionInvariant Images – RGB:

Information from one pixel, with calibrationInformation from all pixels – use entropy

New Multispectral data: Information from one pixel without

calibration, but knowledge of narrowband sensors peak wavelengths

3

RGB Illumination Invariant

4

Removing Shadows from Images, ECCV 2002Graham Finlayson, Steven Hordley, and Mark Drew

-0.5 0 0.5 1 1.5 2 2.5 3-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

log(r/g)lo

g(b

/g)

An example, with delta function sensitivities

400 450 500 550 600 650 7000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Wavelength

Re

lati

ve

Se

ns

itiv

ity

B

W R

YG

PNarrow-band

(delta-function sensitivities)

Log-opponent chromaticities for 6 surfaces under 9 lights

RGB…

Deriving the Illuminant Invariant

-0.5 0 0.5 1 1.5 2 2.5 3-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

log(r/g)

log

(b/g

)

Log-opponent chromaticities for 6 surfaces under 9 lights

This axis is invariant to illuminant colour

Rotate chromaticities

RGB…

Normalized sensitivities of a SONY DXC-930 video

camera

An example with real camera data

400 450 500 550 600 650 7000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Wavelength

Re

lativ

e S

en

siti

vity

Log-opponent chromaticities for 6

surfaces under 9 different lights

RGB…

Deriving the invariant

Log-opponent chromaticities

The invariant axis is now only approximately illuminant

invariant (but hopefully good enough)

Rotate chromaticities

RGB…

Image FormationIllumination : motivate using theoretical

assumptions, then test in practicePlanck’s Law in Wien’s approximation:

Lambertian surface S(), shading is , intensity is I

Narrowband sensors qk(), k=1..31, qk()=(-k)

Specular: colour is same as colour of light (dielectric):

9

Multispectral

To equalize confidence in 31 channels, use a geometric-mean chromaticity:

Geometric Mean Chromaticity:

with

Multispectral Image Formation …

10

{ }

Multispectral Image Formation …

11

sensor-dependent

illumination-dependent

surface-dependent

So take a log to linearize in (1/T) !

Logarithm:

12

Multispectral Image Formation …

known because, in special case of multispectral, *know* k !

Only sensor-unknown is ! ( spectral-channel gains)

klog

If we could identify at least one specularity, we could recover log k ??

Nope, no pixel is free enough of surface colour .So (without a calibration) we won’t get log k, but

instead it will be the origin in the invariant space.Note: Effect of light intensity and shading removed:

31D 30-DNow let’s remove lighting colour too: we know 31-

vector (ek – eM) (-c2/k - c2/M)

Projection to (ek – eM) removes effect of light, 1/T : 30D 29-D

13

Multispectral Image Formation …

Algorithm:

-Form 31-D chromaticity k

- Take log

- Project to (ek – eM) using projector Pe

Algorithm:

What’s different from RGB? For RGB have to get “lighting-change

direction”(ek – eM) either from (i)calibration, or (ii) internal evidence (entropy) in the

image.

For multispectral, we know (ek – eM) !

First, consider synthetic images, for understanding:

16

Camera: Kodak DSC 420

31 sensor gains qk()

Surfaces: 3 spheres, reflectances from Macbeth ColorChecker

Carry out all in 31-D, but show as camera would see it.

Synthetic Images

17

Under red light, P2800

Under blue light, P10500

shading, for light 1, for light 2

Synthetic Images

18

Spectral invariant

Original: not invariant

Measured Multispectral Images

19

Under D75 Under D48

Invt. #1 Invt. #2

Measured Multispectral Images

20

In-shadow, In-light

After invt. processing

Measured Multispectral Images

21

Measured Multispectral Images

22

Measured Multispectral Images

23

ConclusionA novel method for producing illumination

invariant, multispectral image Successful in removing effects of

Illuminant strength, colour, and shading

24

Next: removing shadows from remote-sensing data.

25Multispectral Images Invariant to Illumination Colour, Strength and Shading

Thanks!

Funding: Natural Sciences and Engineering Research Council of Canada