EECS 274 Computer Vision

Sources, Shadows, and Shading

Surface brightness

• Depends on local surface properties (albedo), surface shape (normal), and illumination

• Shading model: a model of how brightness of a surface is obtained

• Can interpret pixel values to reconstruct its shape and albedo

• Reading: FP Chapter5, H Chapter 11

Radiometric properties of sources• How bright (or what color)

are objects?• One more definition:

Exitance of a light source is– the internally generated

power (not reflected) radiated per unit area on the radiating surface

• Similar to radiosity: a source can have both– radiosity, because it

reflects– exitance, because it emits

• Independent of its exit angle

• Internally generated energy radiated per unit time, per unit area

• But what aspects of the incoming radiance will we model?– Point, line, area source– Simple geometry

dPLPE oe 00 cos),,()(

Radiosity due to a point sources

• small, distant sphere radius e and exitance E, which is far away subtends solid angle

2

r

Typo in figure, d r

Radiosity due to a point source

id

id

Er

PB

cos

cos termExitanceangle solid2

• As r is increased, the rays leaving the surface patch and striking the sphere move closer evenly, and the collection changes only slightly, i.e., diffusive reflectance, or albedo

• Radiosity due to source

Nearby point source model

• The angle term can be written in terms of N and S• N is the surface normal• ρd is diffuse albedo• S is source vector - a vector from P to the source, whose

length is the intensity term, ε2E– works because a dot-product is basically a cosine

2

2

cosPr

PSPNPE

rPB did

Point source at infinity• Issue: nearby point source

gets bigger if one gets closer– the sun doesn’t for any

reasonable binding of closer

• Assume that all points in the model are close to each other with respect to the distance to the source

• Then the source vector doesn’t vary much, and the distance doesn’t vary much either, and we can roll the constants together to get:

SPNPPB d )(

2)(

Pr

PSPNPPB d

SN

r

SN

Prr

PSSN

Pr

PSPN

20

0

20

02 ))((

))((

Line sources

radiosity due to line source varies with inverse distance, if the source is long enough

Area sources• Examples: diffuser boxes,

white walls.• The radiosity at a point

due to an area source is obtained by adding up the contribution over the section of view hemisphere subtended by the source – change variables and add

up over the source

Radiosity due to an area source

• ρd is albedo• E is exitance• r is distance

between points Q and P

• Q is a coordinate on the source

source

usi

d

usi

source

d

id

ied

iid

dAr

QEP

r

dAQEp

dQE

P

dQPPLP

dQPPLPPB

2

2

coscos

coscos

cos

cos,

cos,

Shading models• Local shading model

– Surface has radiosity due only to sources visible at each point

– Advantages:• often easy to

manipulate, expressions easy

• supports quite simple theories of how shape information can be extracted from shading

• Global shading model– Surface radiosity is due to

radiance reflected from other surfaces as well as from surfaces

– Advantages:• usually very accurate

– Disadvantage:• extremely difficult to

infer anything from shading values

Local shading models

• For point sources at infinity:

• For point sources not at infinity

Pssd

Pss

SPNP

PBPB

from visiblesources

from visiblesources

)()(

)()(

Ps sd Pr

PSPNPPB

from visiblesources2)(

)()()()(

Shadows cast by a point source• A point that can’t see the

source is in shadow (self cast shadow)

• For point sources, the geometry is simple

Area source shadows

Are sources do not produce darkshadows with crisp boundaries

1. Out of shadow2. Penumbra (“almost shadow”)3. Umbra (“shadow”)

Photometric stereo

• Assume:– A local shading model– A set of point sources that are infinitely

distant– A set of pictures of an object, obtained

in exactly the same camera/object configuration but using different sources

– A Lambertian object (or the specular component has been identified and removed)

Projection model for surface recovery - Monge patch

In computer vision, it is often known as height map , depth map, or dense depth map

Monge patch

Image model• For each point source, we

know the source vector (by assumption)

• We assume we know the scaling constant of the linear camera (i.e., intensity value is linear in the surface radiosity)

• Fold the normal and the reflectance into one vector g, and the scaling constant and source vector into another Vj

• Out of shadow:

• g(x,y): describes the surface

• Vj: property of the illumination and of the camera

• In shadow:

j

j

j

j

Vyxg

kSyxNyx

SyxNyxk

yxkByxI

),(

)(),(),(

)),()(,(

),(),(

0),( yxI j

From many views

• From n sources, for each of which Vi is known

• For each image point, stack the measurements

• Solve least squares problem to obtain g

Tn

T

T

V

V

V

V

2

1

Tn yxIyxIyxIyxi )),(,),,(),,((),( 21

),(

),(

),(

),(

2

1

2

1

yxg

V

V

V

yxI

yxI

yxI

Tn

T

T

n

),(),( yxVgyxi

One linear system per point

Dealing with shadows

Known Known Known Unknown

),(

),(00

0

),(0

00),(

),(

),(

),(

2

1

2

1

2

22

21

yxg

V

V

V

yxI

yxI

yxI

yxI

yxI

yxI

Tn

T

T

nn

),(

),(

),(

),(

2

1

2

1

yxg

V

V

V

yxI

yxI

yxI

Tn

T

T

n

),(),( yxVgyxi

Recovering normal and reflectance

• Given sufficient sources, we can solve the previous equation (e.g., least squares solution) for g(x, y)

• Recall that g(x, y) = r (x,y) N(x, y) , and N(x, y) is the unit normal

• This means that (r x,y) =|g(x, y)|• This yields a check

– If the magnitude of g(x, y) is greater than 1, there’s a problem

• And N(x, y) = g(x, y) / (r x,y)

Five synthetic images

Recovered reflectance

|g(x,y)|=ρ(x,y): the value should be in the range of 0 and 1

Recovered normal field

Recovering a surface from normals• Recall the surface is

written as

• Parametric surface

• This means the normal has the form:

• If we write the known vector g as

• Then we obtain values for the partial derivatives of the surface:

)),(,,( yxfyx

11

1),(

22 y

x

yx

f

f

ffyxN

),(

),(

),(

),(

3

2

1

yxg

yxg

yxg

yxg

),(/),(),(

),(/),(),(

32

31

yxgyxgyxf

yxgyxgyxf

y

x

yx

yx

rrN

ky

fjrk

x

fir

,

Recovering a surface from normals (cont’d)• Recall that mixed second partials are equal --- this gives us a

check. We must have:

(or they should be similar, at least)• We can now recover the surface height at any point by

integration along some path, e.g.

x

yxgyxg

y

yxgyxg

)),(/),(()),(/),(( 3231

v u

xy cdxvxfdyyfvuf0 0

),(),0(),(

Recovered surface by integration

v u

xy cdxvxfdyyfvuf0 0

),(),0(),(

The Illumination Cone

N-dimensional Image Space

x1

x2

What is the set of n-pixel images of an object under all possible lighting conditions (at fixed pose)? (Belhuemuer and Kriegman IJCV 99)

xn

Single light source image

The Illumination Cone

N-dimensional Image Space

x1

x2

What is the set of n-pixel images of an object under all possible lighting conditions (but fixed pose)?

Illumination Cone

Proposition: Due to the superposition of images, the set of images is a convex polyhedral cone in the image space.

xn

Single light source images:Extreme rays of cone

2-light source image

For Lambertian surfaces, the illumination cone is determined by the 3D linear subspace B(x,y), where

When no shadows, then Use least-squares to find 3D linear subspace, subject to the

constraint fxy=fyx (Georghiades, Belhumeur, Kriegman, PAMI, June, 2001)

Original (Training) Images

a(x,y) fx(x,y) fy(x,y) albedo (surface normals) Surface. f (x,y) (albedo

textured mapped on surface)

3D linear subspace

Generating the Illumination Cone

i

ii

i syxBsyxnyxyxI )0,),(max()0,),(),(max(),(

i

isyxByxI ),(),(

Single Light Source Face Movie

Image-based rendering: Cast Shadows

Yale face database B

• 10 Individuals• 64 Lighting Conditions• 9 Poses => 5,760 Images

Variable lighting

Curious experimental fact

• Prepare two rooms, one with white walls and white objects, one with black walls and black objects

• Illuminate the black room with bright light, the white room with dim light

• People can tell which is which (due to Gilchrist)

• Why? (a local shading model predicts they can’t).

Global shading models

A view of a black room with black objects. We see a cross-section of the image intensity corresponding to the line drawn on the image.

A view of a white room with white objects. We see a cross-section of the image intensity corresponding to the line drawn on the image.

Can adjust so that local shading model predicts these pictures will be indistinguishable

What’s going on here?

• Local shading model is a poor description of physical processes that give rise to images– because surfaces reflect light onto one another

• This is a major nuisance; the distribution of light (in principle) depends on the configuration of every radiator; big distant ones are as important as small nearby ones (solid angle)

• The effects are easy to model• It appears to be hard to extract

information from these models

Interreflections - a global shading model• Other surfaces are now area sources

- this yields:

• Vis(x, u) is 1 if they can see each other, 0 if they can’t

usi

d dAuxVisuxr

B(u)xxExB ),(),(

coscos)()()(

surfacesother todueRadiosity Exitance surface aat Radiosity

2

sourcesother all

What do we do about this?

• Attempt to build approximations– Ambient illumination

• Study qualitative effects– reflexes– decreased dynamic range– smoothing

• Try to use other information to control errors

Ambient illumination

• Two forms– Add a constant to the radiosity at every point

in the scene to account for brighter shadows than predicted by point source model• Advantages: simple, easily managed (e.g. how would

you change photometric stereo?)• Disadvantages: poor approximation (compare black

and white rooms– Add a term at each point that depends on the

size of the clear viewing hemisphere at each point• Advantages: appears to be quite a good

approximation, but jury is out• Disadvantages: difficult to work with