View
224
Download
4
Category
Tags:
Preview:
Citation preview
Understanding the effect of lighting in images
Ronen Basri
Light field
• Rays travel from sources to objects• There they are either absorbed or reflected• Energy decreases with distance and number of
bounces• Camera captures the set of rays that travel
through the focal center
Specular reflectance (mirror)
• When a surface is smooth light reflects in the opposite direction of the surface normal
Specular reflectance
• When a surface is slightly rough the reflected light will fall off around the specular direction
Diffuse reflectance
• When the surface is very rough light may be reflected equally in all directions
Diffuse reflectance
• When the surface is very rough light may be reflected equally in all directions
BRDF
• Bidirectional Reflectance Distribution Function
• Specifies for a unit of incominglight in a direction how much light will be reflectedin a direction
�̂�
Lambertian reflectance
Lambert’s law
�̂��̂�𝜃
Preliminaries
• A surface is denoted • A point on is • The tangent plane is spanned by
• The surface normal is given by
Photometric stereo
• Given several images of the a lambertian object under varying lighting
• Assuming single directional source
𝑀=𝐿𝑆
Photometric stereo
• We can solve for S if L is known (Woodham)• If L is unknown we can use SVD factorization
(Hayakawa)
1 1 111 1
1 2
1 2
1 2
3
1 3
...
. ....
. ....
. ....
p x y z
x x
y y
z z pf f f
f fp x y zf p f
I I l l l
n n
n n
n n
I I l l l
𝑀=𝐿𝑆
Factorization• Use SVD to find a rank 3 approximation
• Define
• Factorization is not unique, since
invertible
To reduce ambiguity we impose integrability –up to generalized bas relief transformation (Belhumeur et al.)
𝑀=𝑈 Σ𝑉 𝑇
�̂�=𝑈 √ Σ , �̂�=√Σ𝑉 𝑇 and �̂�= �̂��̂�
�̂�=( �̂� 𝐴−1 ) ( 𝐴�̂� ) , 𝐴
Integrability
• Recall that
• Given , we set
• And solve for
Shape from shading (SFS)
• What if we only have one image?• Assuming that lighting is known
and uniform albedo
• Every intensity determinesa circle of possible normals
• There is only one unknown ()if uniform albedo is assume
𝐼=𝐸 𝜌 cos𝜃
Shape from shading
• We write
• Therefore
• We obtain
• This is a first order, non-linear PDE (Horn)
Shape from shading
• Suppose , then
• This is called an Eikonal equation
Distance transform
• The distance of each point to the boundary• Posed as an Eikonal equation
Solution
• Right hand side in SFS
determines “speed”• Eikonal equation can be solved by a
continuous analog of “shortest path” algorithm, called “fast marching”
• The case is handled by change of variables (Kimmel & Sethian)
Example
Illumination cone• What is the set of images of an object under
different lighting, with any number of sources? • Due to additivity, this set forms a convex cone in
number of pixels (Belhumeur & Kriegman)
= 0.5* +0.2* +0.3*
Illumination cone
• Cone characterization is generic, holds also with specularities, shadows and inter-reflections
• Unfortunately, representing the cone is complicated (infinite degrees of freedom)
• Cone is “thin” for Lambertian objects;indeed the illumination cone of many objects can be represented with few PCA vectors (Yuille et al.)
Lambertian reflectance is smooth
0 1 2 30
0.5
1
0 1 2 30
0.5
1
1.5
2
lighting
reflectance
(Basri & Jacobs; Ramamoorthi & Hanrahan)
Reflectance obtained with convolution
+++
Reflectance obtained with convolution
+++
Spherical harmonics
1
Z YX
23 1Z XZ YZ22 YX XY
2 2 2 1X Y Z Positive values
Negative values
Harmonic approximation
• Lighting, in terms of harmonics
• Reflectance
• Approximation accuracy, 99%(Basri & Jacobs; Ramamoorthi & Hanrahan)
Harmonic transform of kernel
1.023
0.495
-0.111
0.05
-0.029
0.886
-0.5
0
0.5
1
1.5
0 1 2 3 4 5 6 7 8
𝑘(𝜃)=max ¿
“Harmonic faces”
Positive values
Negative values
( , , )x y zn n n n
ρ Albedo
n Surface normal
2(3 1)zn 2 2( )x yn n x yn n x zn n y zn n
r
zn xn yn
Photometric stereo
LM
S
Image n
:
Image 1
Light n
:
Light 1
SVD recovers L and S up to an ambiguity
r
rnz
rnz
rny
(3r nz2-1)
(r nx2-ny2)
rnxny
rnxnz
rnynz
r r
(Basri, Jacobs &Kemelmacher)
Photometric stereo
Motion + lighting
Motion + lighting
• Given 2 images
• Take ratio to eliminate albedo
• If motion is small we can represent using a Taylor expansion around
(Basri & Frolova)
Small motion
• We obtain a PDE that is quasi linear in
• Where
with
• Can be solved with continuation (characteristics)
Reconstruction
More reconstructions
Conclusion
• Understanding the effect of lighting on images is challenging, but can lead to better interpretation of images
• We surveyed several problems:– Photometric stereo– Shape from shading– Modeling multiple sources with spherical harmonics– Motion and lighting
• We only looked at Lambertian objects…
Recommended