Modeling Lambertian Surfaces Under Unknown Distant Illumination UsingHemispherical Harmonics

Shireen Elhabian, Ham Rara, Aly FaragECE Dept, CVIP Lab

University of LouisvilleLouisville, KY, USA

syelha01,hmrara01,aafara01@louisville.edu

Abstract—A surface reflectance function represents the pro-cess of turning irradiance signals into outgoing radiance.Irradiance signals can be represented using low-order basisfunctions due to their low-frequency nature. Spherical har-monics (SH) have been used to provide such basis. Howeverthe incident light at any surface point is defined on theupper hemisphere; full spherical representation is not needed.We propose the use of hemispherical harmonics (HSH) tomodel images of convex Lambertian objects under distantillumination. We formulate and prove the addition theoremfor HSH in order to provide an analytical expression of thereflectance function in the HSH domain. We prove that theLambertian kernel has a more compact harmonic expansionin the HSH domain when compared to its SH counterpart. Wepresent the approximation of the illumination cone in the HSHdomain where the non-negative lighting constraint is forced.Our experiments illustrate that the 1st order HSH outperforms1st and 2nd order SH in the process of image reconstructionas the number of light sources grows. In addition, we provideempirical justification of the nonnecessity of enforcing the non-negativity constraint in the HSH domain, allowing us to useunconstrained least squares which is faster than the constrainedone, while maintaining the same quality of the reconstructedimages.

Keywords-hemispherical harmonics, spherical harmonics,Legendre polynomials, illumination modeling.

I. INTRODUCTION

Image reconstruction under unknown arbitrary lighting isa fundamental process of many computer vision tasks in-cluding shape recovery and object recognition. One possibleway to tackle such a problem is to use low-dimensionallinear subspace to represent a certain class of images, wheresuch representation should capture variations due to differentimaging conditions. As outlined by Lee et al. [1], thissubspace can be obtained using three different ways; (1)performing Principal Component Analysis (PCA) on a largeset of images of object(s) of interest under different imagingconditions, (2) using 3D models, image formation processcan be simulated to render synthetic images under assumedimaging conditions, while PCA is again used to computethe required subspace, or (3) under the assumption ofLambertian surface, basis images can directly be constructedusing harmonic expansion of the lighting function. In this

Figure 1. The incoming light from the surrounding environment at asurface point x is only defined on the upper hemisphere Ω+ oriented bythe surface normal at this point, where the information contained in theinner hemisphere is never used.

paper, we investigate a hemispherical harmonic space torepresent images of Lambertian objects under unknowndistant lighting.

An image produced by a convex Lambertian surface is analbedo-modulated version of the surface reflectance function,which is an analytic expression that represents the processof turning the incident radiance (irradiance) into outgoingradiance (reflection). The surface irradiance signal has alow frequency nature, which allows for its representationusing low-order basis functions [2]. Basri and Jacobs [3]and Ramamoorthi [4] formulated the surface reflectancefunction in a convolution framework where the lightingfunction acts as a signal filtered by the Lambertian kernel.They provided an analytical expression of an image of aconvex-Lambertian object illuminated by distant lighting.They used spherical harmonics (SH) to represent the surfacereflectance function in the frequency domain, under theassumption that both the light and the surface bidirectionalreflectance distribution function (BRDF) are spherical func-tions. However, the incoming light from the surroundingenvironment at any surface point is only defined on theupper hemisphere, oriented by the surface normal at thispoint [5], see Fig. 1. Representing hemispherical functions inthe spherical domain presents discontinuities at the boundaryof the hemisphere [5], hence using spherical basis demands

more coefficients to represent such functions. Thus the keyissue is how to efficiently and accurately represent suchhemispherical functions.

Several hemispherical basis have been proposed in lit-erature to represent hemispherical functions. Sloan et al.[6] used SH to represent an even-reflected version of ahemispherical function. Coefficients were found using leastsquares SH, however this lead to non-zero values in thelower hemisphere. Koenderink et al. [7] used Zernike poly-nomials [8], which are basis functions defined on a disk,to build hemispherical basis. Yet, such polynomials havehigh computational cost. Makhotkin [9] and Gautron etal. [5] proposed hemispherically orthonomal basis throughmapping the negative pole of the sphere to the border of thehemisphere. Such contraction was achieved through shiftingthe adjoint Jacobi polynomials [9] and the associated Leg-endre polynomials [5] without affecting the orthogonalityrelationship. Habel and Wimmer [2] used the SH as anintermediate basis to define polynomial-based hemisphericalbasis denoted by H-basis. They used the SH basis functionswhich are symmetric to the z = 0 plane since they areorthogonal over the upper hemisphere. While other basisfunctions are shifted the same way proposed by [5]. Al-though such basis definition lead to polynomial basis, thisinhibit us from deriving an analytical expression of harmonicexpansion of the Lambertian kernel.

In this paper, we adopt the hemispherical basis definedby Gautron et al. [5]. Our contributions can be outlined asfollows: (1) 1st order hemispherical harmonic (HSH) basisand (2) a closed form expression for HSH expansion forLambertian kernel are derived, (3) the addition theorem forHSH is formalized and proved, (4) the application of HSHbasis to image reconstruction under unknown distant lightingis investigated, and (5) empirical justification of the non-necessity of enforcing the non-negative lighting constraint inthe HSH domain is presented, yeilding unconstrained leastsquares minimization.

The rest of the paper is organized as follows: Section2 discusses hemispherical harmonics. Section 3 talks aboutthe surface reflectance function. Section 4 and 5 provide theformulation of image formation and reconstruction processesin HSH domain. Section 6 derives the non-negative lightingconstraint in case of HSH approximation. Later sections dealwith the experimental results and conclusions.

II. HEMISPHERICAL HARMONICS

Makhotkin [9] outlined a generic approach for definingharmonic basis functions, where the Legendre polynomialsPn(z) are chosen, then differentiated m-times to defineassociated Legendre polynomials Pmn (z) which are definedby the differential equation (1) [3], and eventually theharmonics basis functions are constructed by combining

Pmn (z) with cos(mφ), sin(mφ).

Pmn (z) =(1− z2)m/2

2nn!

dm+n

dzm+n(z2 − 1)n (1)

with order n ≥ 0 and degree m ∈ [0, n]. Gautron etal [5] defined hemispherical basis functions as an adaptedversion of spherical basis by shifting the associated Legendrepolynomials, which remains orthogonal but with differentnormalization factor. The main idea behind this shifting islimiting the domain of the elevation angle θ to [0, π2 ] whensubstituting z by cos θ.

Pmn (z) = Pmn (2z − 1) (2)Hemispherical basis functions Hm

n (θ, φ) are definedfrom the shifted associated Legendre polynomials as follows[5],

Hmn (θ, φ) =

√2Nm

n Pmn (cos θ) cos(mφ) m > 0

N0nP

0n(cos θ) m = 0

√2Nm

n P−mn (cos θ) sin(−mφ) m < 0

(3)

where n ≥ 0 , m ∈ [−n, n], θ ∈ [0, π2 ] , φ ∈ [0, 2π] and

Nmn =

√(2n+ 1)(n− |m|)!

2π(n+ |m|)! (4)

The 1st order hemispherical basis functions representedin cartesian coordinates can be written as follows where thesuperscripts e and o denote the even and odd componentsof the basis functions.

H00 = 1√2π

Ho11 = 2

√3

2π

y√z(1−z)√x2+y2

H10 =√

32π (2z − 1) He

11 = 2√

32π

x√z(1−z)√x2+y2

(5)

The hemispherical basis functions form a complete setof basis for hemispherical function approximation over the[0, π/2]× [0, 2π] domain. Thus they can be used to expandthe lighting function and the surface BRDF.

III. SURFACE REFLECTANCE FUNCTION

The surface reflectance function describes the radianceof all possible surface normals given the light distribution,where the radiance leaving a surface at a point x due toirradiance received over the whole incoming hemisphereΩ+, i.e. irrespective of the incoming direction, can beobtained by integrating over contributions/irradiances fromall incoming directions. Consider the incoming/upper hemi-sphere represented in spherical domain Ω+ = θ, φ : θ ∈[0, π/2], φ ∈ [0, 2π]. A surface at a point x illuminatedby radiance L(x,ui) coming in from an incident directionalregion of solid angle dωi at incident direction ui = (θi, φi)receives irradiance (lighting function) L(x,ui) cos θidωi.Under the assumption of distant light source, all surfacepoints receive the same amount of light, hence we can omit

the positional variance of the lighting function. The sur-face Bidirectional Reflectance Distribution Function (BRDF)ρ(ui,uo) describes the amount of incident light from theincident direction ui being reflected by the surface at certainoutgoing direction uo. Thus the reflectance function can beobtained by integrating irradiance-weighted BRDF over allincoming directions,

R(n(x),uo) =

∫Ω+

L(ui)ρ(ui,uo) cos θidωi

=

∫Ω+

L(ui)ρ(ui,uo)(ui.n(x))dωi (6)

Fixing the outgoing direction at the north pole, the re-flectance can be rewritten as,

R(n(x)) =

∫Ω+

L(ui)K(ui.n(x))dωi (7)

Noting that the BRDF of Lambertian surfaces is constant(1/π) since they reflect light equally in all directions,such constant is ignored because it only scales the imageintensities by a constant factor. In addition, the cosine termis absorbed in the kernel K(.) which encodes a radiallysymmetric clamped BRDF. In case of Lambertian surfaces,K(ui.n(x)) = ui.n(x) = cos θi. For a given surfacenormal, the integration in (7) over all incident directions isanalogous to a convolution [3], where the lighting functionis considered as a signal being filtered by the surfacereflectance kernel.

In order to express (7) in HSH domain, we need toformulate the addition theorem of HSH, refer to Appendix Afor proof.

Addition theorem for HSH. Let u1 : (θ1, φ1) and u2 :(θ2, φ2) be two distinct unit directions in the standardspherical coordinates system (x, y, z) separated by angleγ such that [10],

u1.u2 = cos γ = cos θ1 cos θ2 + sin θ1 sin θ2 cos(φ1 − φ2)

with θ1, θ2 ∈ [0, π2 ] and φ1, φ2 ∈ [0, 2π]. The additiontheorem for hemispherical harmonics asserts that,

Pn(cosγ) =2π

2n+ 1

n∑m=−n

Hmn (θ1, φ1)Hm∗

n (θ2, φ2) (8)

A. Reflectance Function Of Lambertian SurfacesConsider the reflectance function in (7), since the surface

reflectance kernel K(ui.n(x)) is radially symmetric, i.e.there is no azimuthal dependence, it can be expanded usingzonal harmonics Pn(ui.n(x)), centered at surface normaln(x), where m is set to 0.

K(ui.n(x)) =

∞∑n=0

knPn(ui.n(x)) (9)

Using the addition theorem of HSH, (9) can be written as,K(ui.n(x)) =

∞∑n=0

2π

2n+ 1kn

n∑m=−n

Hmn (n(x))Hm∗

n (ui) (10)

By expanding the incident radiance L(ui) in HSH,

L(ui) =

∞∑n′=0

n′∑m′=−n′

ln′m′Hm′

n′ (ui) (11)

Using the orthonormality property of hemispherical basis[5], where all terms vanish except for n′ = n and m′ = m,we can rewrite the reflectance function as follows,

R(n(x)) =

∞∑n=0

n∑m=−n

lnmαnHmn (n(x)) (12)

where,αn =

2π

2n+ 1kn (13)

B. HSH For Lambertian Kernel Approximation

In general, BRDFs are functions defined over the cartesianproduct of the incoming and outgoing hemispheres [5].Nonetheless, a Lambertian surface is perceived in the sameway regardless of the outgoing/viewing direction, that is whythe Lambertian kernel is a function of the incident directiononly. Basri et. al [3] derived the SH expansion of such kernel,proving that the second order expansion retains 99.96% ofthe kernel’s energy. However, the surface reflectance kernel(clamped BRDF) is a hemispherical function, maintainingall of its information in the upper hemisphere. Thus usinga spherical representation will lead to inefficient use of theharmonic coefficients. In this work, we prove analyticallythat we only need first order hemispherical harmonic ex-pansion to accurately represent the Lambertian kernel.

The hemispherical harmonic expansion of the Lambertiankernel can be derived using the zonal harmonics and theseries representation of the shifted Legendre polynomials[10] leading to, (proof is omitted due to lack of space)

kn = (−1)n√

2π(2n+ 1)

n∑k=0

(−1)k(n+ k)!

(k + 2)(k!)2(n− k)!(14)

where the first few coefficients are k0 ≈ 1.2533, k1 ≈0.7236 and kn ≈ 0 ∀n ≥ 2, these values are verified usingMonte Carlo integration with hemispherical sampling. Fig. 2shows the graphical representation of the first nine coeffi-cients in case of spherical and hemispherical harmonics. Itcan be observed that the HSH coefficients of the Lambertiankernel decay quickly in the HSH domain, faster than that ofthe SH domain. The energy [3] captured by each componentis concentrated in the HSH domain when compared to thatof the SH domain. While the cumulative energy saturates to100% using the HSH 1st order approximation, while 99.22%is achieved using the SH 2nd order approximation, seeTable I. Thus it can be concluded here that the Lambertiankernel is dominated by the first two coefficients in the HSHdomain compared to the first three coefficients in the SHdomain. Thus high frequency light components will havelittle effect on the reflectance function, allowing for itslow-dimensional approximation. Fig. 3 shows a 2D sliceof the Lambertian kernel, both spherical and hemispherical

(a) Spherical harmonic expansion coefficients, reproduced from [3]for comparison

(b) Hemispherical harmonic expansion coefficients

Figure 2. From left to right; a graphical representation of the firstnine harmonic expansion coefficients of the Lambertian kernel, the relativeenergy maintained by each coefficient and the cumulative energy. Note thatthe Lambertian kernel acts as a low pass filter, where cut-off frequency inthe HSH domain is less than that of the SH domain, allowing lower numberof components to represent the Lambertian kernel. The actual values aretabulated in Table I

Table ITHE RELATIVE AND CUMULATIVE ENERGY OF ZONAL SH AND

HSH

Order (n) 0 1 2SH Energy 37.5% 50% 11.72%SH Cumm. Energy 37.5% 87.5% 99.22%HSH Energy 75% 25% 0%HSH Cumm. Energy 75% 100% 100%

harmonics are used for kernel approximation. The 1st orderapproximation is enough for HSH to provide a good ap-proximation to the Lambertian kernel, while the 1st SH doesnot capture enough information for function approximation.Fig. 4 shows the mean square error of the Lambertian kernelapproximation as a function of the number of coefficientsused, where the hemispherical harmonics expansion providesbetter function reconstruction at lower order when comparedto the spherical harmonics reconstruction. Another functionapproximation performance metric is the accuracy measureused by [5], which computes the fraction of the total energycaptured for a given number of coefficients. It is evidentthat hemispherical harmonic provides better representationfor Lambertian kernel with fewer number of coefficients.

C. Reflectance Function Approximation

According to the HSH addition theorem, the lightingfunction harmonic expansion is modulated by a scaling

(a) (b)

Figure 3. A Lambertian kernel slice (solid blue) and its first orderapproximation (dashed red) using (a) spherical harmonics (b) hemisphericalharmonics

(a) (b)

Figure 4. (a) Reconstruction mean square error (MSE) and (b) Accuracy,versus number of coefficients used for Lambertian kernel approximation

factor αn which depends on the expansion order n ofthe Lambertian kernel. This emphasis that the lightinghigh frequency components have insignificant impact onthe reflectance function [3][4]. Since the Lambertian kernelis dominated by the 1st order approximation in the HSHdomain, we can truncate (12) as,

R(n(x)) ≈1∑

n=0

n∑m=−n

lnmRmn (n(x)) (15)

with Rmn (n(x)) = αnHmn (n(x)). Having 2n+ 1 harmonics

for a given order n, the 1st order approximation will leadto an approximation of the reflectance function using fourharmonic basis.

IV. IMAGE FORMATION

The reflectance function describes the reflectivity of aunit-albedo sphere under arbitrary lighting function. The im-age produced by a convex Lambertian surface with spatiallyvarying albedo is formed by allowing each surface point toinherit its intensity from that point on the sphere having thesame normal, such intensity is scaled by the point’s albedo.Formally, let xj be the jth point on the surface with normaln(xj) and albedo ρ(xj). Using the N th order harmonicapproximation of the reflectance function (15), the imageof this point can be obtained by,

I(xj) = ρ(xj)R(n(xj))

≈ ρ(xj)

N∑n=0

n∑m=−n

lnmRmn (n(xj))

=

N∑n=0

n∑m=−n

lnm (ρ(xj)Rmn (n(xj)))

=

N∑n=0

n∑m=−n

lnmbmn (ρ(xj),n(xj))

=

N(N+2)∑k=0

lkbk(ρ(xj),n(xj)) (16)

Therefore, any image of an object under varying arbitrarylighting function can be efficiently approximated as a linearcombination of harmonic basis images of the form,

bk(ρ(xj),n(xj)) = ρ(xj)Rk(n(xj)) (17)

with k = n(n + 1) + m. In matrix notation; Let Bdenotes p × (N + 1)2 matrix whose columns are theharmonic basis images bk(n(xj)) where i ∈ [1, p]. Let` = (l0, l1, ..., lN(N+2))

T denote the lighting coefficients.Thus (16) can be rewritten as follows,

I ≈ B ` (18)

V. IMAGE RECONSTRUCTION

As a consequence of Fig. 2(b), the first four harmonicimages are enough to reconstruct any image under arbitrarylighting, i.e. I(xj) =

∑3k=0 lkbk(ρ(xj),n(xj)), where for

a given surface point of albedo ρ and surface normal n =(nx, ny, nz), we have,b0(ρ,n) = c0ρ b1(ρ,n) = c1ρ

ny√nz(1−nz)√n2x+n2

y

b2(ρ,n) = c2ρ(2nz − 1) b3(ρ,n) = c3ρnx√nz(1−nz)√n2x+n2

y

(19)where n2

x = nx × nx, n2y = ny × ny and ck are constants

depending on α and lambertian kernel coefficient k of thecorresponding order.

Given an image under unknown distant lighting, the mainconcern is to recover the coefficients lk which characterizehow the harmonic basis can be combined to reconstruct theinput image. Eq. (18) lends itself to an over-determinedlinear system of equations, since the number of equationsp is much greater than the number of unknowns (= 4).The minimal solution can be obtained using Singular ValueDecomposition (SVD)1, i.e. = VS−1UT I , where B =USVT . The first four columns of U are used, and S is 4×4.After computing the coefficient vector , the reconstructedimage I is solved using I = B. These step are enumeratedin Algorithm 1.

Algorithm 1 HSH-based image reconstruction1: Input: (a) Image to be reconstruct, I , (b) Matrix of

harmonic basis images, B2: Output: Reconstructed image, I3: Compute the SVD decomposition of the matrix of basis

images B: [U,S,V] = svd(B)4: Retain the first n-columns of U: U = U(:, 1 : n), wheren is the number of harmonic basis images

5: Retain the first n-singular values in S: S = S(1 : n, 1 :n)

6: Solve for the HSH coefficient vector using the least-squares estimate of I = B ` where = VS−1UT I

7: Compute the reconstructed image; I = BFig. 5 illustrates the HSH-based image reconstruction

procedure, where the harmonic basis images are computed

1SVD, compared to QR-decomposition, is relatively slower but moreaccurate, having the ability to handle rank-deficient matrices. Surface pointswith attached shadows, not receiving enough light, will have low responsein the harmonic domain, leading to zero-rows in the basis matrix B, thusthe matrix B might not be a full-rank matrix.

Figure 5. Illustration for Hemispherical harmonics based image reconstruc-tion, the input image is reconstructed by projecting it first to the 1st fourharmonic basis images (i.e., solve for the coefficients) and then taking thereverse process by summing the scaled basis images (using the computedcoefficients). The reconstructed image is visually similar to the input image.

from the object’s shape (i.e. surface normals) and albedo.Notice that the reconstructed image is visually similar to theinput image. The figure shows the four HSH-based basisderived from a sample facial surface with known surfacenormals and albedo. Note that the first basis image showsthe DC component which describes the appearance of theobject under ambient lighting.

VI. HSH-BASED ILLUMINATION CONE APPROXIMATION

Equation (16) accounts for arbitrary combination of har-monic basis images, controlled by the lighting functioncoefficients lk. However, this might result in non-realizableimages due to negative lighting coefficients [3]. Belhumeurand Kriegman [11] has concluded that a convex cone can beused to represent all possible image produced by an objectunder non-negative lighting, they call this the illuminationcone. Basri and Jacobs [3] have provided approximations tothis cone in the space spanned by the spherical harmonicbasis images. They used the projection of the illuminationcone onto the the low-dimensional SH space to enforce thenon-negative lighting constraint. In this paper, we extend theillumination cone approximation in case of hemisphericalharmonic basis.

A single non-negative directional/distant light source canbe described by a delta function in spherical coordinates[4]. Let ul = (θl, φl) denote the angular coordinates of thelight source, then the lighting function as a function of theincident direction can be written as,L(ui) = L(θi, φi) = δ(θi − θl)δ(φi − φl) = ∆θlφl (20)

The hemispherical harmonic transform of the delta functioncan be written as,

δnm|θlφl =

∫Ω+

L(ui)Hm∗n (ui)dωi

=

∫ 2π

0

∫ π2

0

L(θi, φi)Hm∗n (θi, φi) sin θidθidφi

=

∫ 2π

0

∫ π2

0

δ(θi − θl)δ(φi − φl)

Hm∗n (θi, φi) sin θidθidφi

= Hm∗n (θl, φl) (21)

According to (11), the hemispherical harmonic expansion ofthe delta function is given by,

∆θlφl =

∞∑n=0

n∑m=−n

δnm|θlφl Hmn (θi, φi)

=

∞∑n=0

n∑m=−n

Hm∗n (θl, φl)H

mn (θi, φi) (22)

Having M light sources, the non-negative lighting functioncan be represented as a non-negative combination of deltafunctions as [3],

L(θi, φi) =

M∑l=1

al∆θlφl

=

M∑l=1

al

∞∑n=0

n∑m=−n

Hm∗n (θl, φl)H

mn (θi, φi)

(23)

where al,≥ 0 ∀ l ∈ [1,M ]. Using the linearity of theharmonic transform [3], and compare with (11), the lightingfunction tranform lnm is a non-negative combination of theharmonic transform of the delta functions having the sameorder n and degree m,

L(θi, φi) =

∞∑n=0

n∑m=−n

(M∑l=1

alHm∗n (θl, φl)

)Hmn (θi, φi)

=

∞∑n=0

n∑m=−n

lnmHmn (θi, φi) (24)

Using (16) and the linearity of the harmonic transform, theimage formed using this non-negative lighting function canbe expressed as a non-negative combination given by,

I(xj) ≈N∑n=0

n∑m=−n

(M∑l=1

alHm∗n (θl, φl)

)bmn (n(xj))

=

M∑l=1

al

N∑n=0

n∑m=−n

Hm∗n (θl, φl)b

mn (n(xj))

=

M∑l=1

al

N(N+2)∑k=0

H∗k(ul)bk(n(xj)) (25)

where N defines the order of approximation and k =n(n + 1) + m. Let B denotes p × (N + 1)2 matrix whosecolumns are the harmonic basis images bk(n(xj)) wherei ∈ [1, p]. Let H denotes M × (N + 1)2 matrix whosecolumns contain the sampling of the harmonic functionsHk(ul) over a hemisphere where l ∈ [1,M ] (we use MonteCarlo sampling), while its rows contain the hemisphericalharmonic transform of the delta functions at different light-ing directions. Note that the conjugate is dropped due toreal harmonic functions. Let a = (a1, a2, ..., aM )T denotethe non-negative coefficients. Thus in matrix notation, (25)can be rewritten as follows,

I ≈ BHTa (26)

Given an image under unknown arbitrary lighting, themain concern is to recover the non-negative coefficientsal which characterize the lighting function, this can beaccomplished through solving the non-negative least squaresproblem;

mina‖BHTa− I‖ s. t. a ≥ 0 (27)

where HTa is a (N + 1)2 × 1 vector that represents theN th order harmonic expansion of the lighting function, thusBHTa is the object’s image under this lighting.

VII. EXPERIMENTAL RESULTS

In order to assess 1st order HSH-based image recon-struction versus using 1st and 2nd order SH, we use USFHumanID 3D human face database [12] which contains 100subjects of diverse gender and ethnicity. Each 3D facialsurface is represented as a 2-manifold triangular mesh :G = (V,F), where V = v1, v2, ..., vJ is a set ofJ−vertices with vj ∈ R3 and F = t1, t2, ..., tJ′ is a setof J ′−triangular faces, with the ith face ti = vi0, vi1, vi2is constructed from three vertices with indices i0, i1 and i2.Each vertex is associated with albedo in the red-green-bluechannels, ρr(vj), ρg(vj) and ρb(vj). We use orthographicprojection to re-represent facial triangular meshes in termsof Monge patches which suggests representing the surfaceas (xj , f(xj)) where xj = (xj , yj) and j ∈ [1, p]. Weconsider this representation is attractive since it providesa bijective mapping between surface points and imagecoordinates. Each surface point is associated with a gray-scale albedo ρ = 0.3ρr + 0.59ρg + 0.11ρb. We use forwardfinite difference to approximate surface derivatives to obtainp(x) = ∂f(x)/∂x and q(x) = ∂f(x)/∂y. Thus the surfacenormal can be obtained as,

n(xj) = (nx(xj), ny(xj), nz(xj))T

=(−p(xj),−q(xj), 1)T√p2(xj) + q2(xj) + 1

(28)

Consequently, hemispherical harmonic basis images are con-structed using (19) to be used in image reconstruction.

In order to quantitatively analyze the performance ofthe proposed image model, we simulate image formationprocess by fixing the camera and the surface in position,and illuminating the facial surface using a combination ofdirectional/distant light sources. Adopting a local shadingmodel with no ambient illumination and assuming Lamber-tian surface, the image is constructed by,

I(xj) = ρ(xj)

∫Ω+

L(ui)K(ui.n(xj))dωi (29)

To test the effect of the number of light sources usedto illuminate the surface, we run image formation andreconstruction experiments for the USF subjects using 1to 30 light sources with directions ui randomly sampledusing Monte Carlo sampling, where lights are cast with equal

Figure 6. Themean and standarddeviation of themean absolute errormeasured betweenthe ground-truthimages and thereconstructed ones(using 1st orderHSH compared to1st and 2nd orderSH) as a functionof number of lightsources.

probabilities. For a specific number of light sources and adatabase subject, the experiment is repeated 100 times toguarantee the statistical significance of our results. For eachsynthetic/formed image I , the recovered/reconstructed imageI is compared with the groundtruth image I by computingthe mean and the standard deviation of the absolute error.Fig. 6 shows reconstruction performance as a function ofnumber of light sources for 1st order HSH versus 1st and2nd order SH. Using more light sources, the distributionof intensity becomes independent of light direction, whichmakes the net effect resembles (0,0,1), i.e. frontal, lightsource which is easy to reconstruct. This explains theasymptotic plateau reached by the 1st order HSH, 1st and2nd order SH. According to Fig. 6, the light source cannot bedescribed sufficiently with the four spherical basis images,thus the 1st order SH provides the worst reconstructionperformance when compared to 1st order HSH and 2nd orderSH. Using more spherical basis, 2nd order SH performanceseems to be stable regardless of the number of light sources.On the other hand, 1st HSH provides a midway performancefor low number of sources and exponentially surpass theother two as the number of sources grows. We can concludethat, with more distant light sources, HSH presents a lower-dimensional space for modeling images under unknownarbitrary lighting function.

In order to enforce the non-negative lighting constraint,we used optimized version of the Lawson & Hanson ap-proach [13][14] for solving the non-negative least squaresminimization in (27). Fig. 7 plots the mean absolute errorof image reconstruction with and without enforcing the non-negativity constraint. It can be observed that enforcing suchconstraint is needed in case of SH approximation due tolower mean absolute error. On the other hand, this constraintdoes not have a significant impact on the HSH approxima-tion. In other words, we can have the same quality of imagereconstruction with or without this constraint, which is atthe same time better reconstruction when compared to theSH case. As noted by Bro and Jong [14], the time requiredfor solving least squares problems under the non-negativityconstraint is typically many times longer than for estimating

Figure 7. Enforcingthe non-negativelighting constraint.The mean absoluteerror measuredby comparingthe ground-truthimages and thereconstructed onesas a function ofnumber of lightsources (NN meanswith enforcing non-negative lightingconstraint).

an unconstrained problem. This adds to the benefits of usingHSH approximation in modeling images of Lambertianobjects, where we can use unconstrained model to inferlighting coefficients, solved using SVD, rather than usingtime-consuming non-negative least squares optimization.

VIII. CONCLUSION

In this paper, we introduced the use of hemisphericalharmonics for modeling images of convex-Lambertian sur-faces under unknown distant lighting. The closed formexpression of the Lambertian kernel emphasized that it actsas a low pass filter with lower cut-off frequency in the HSHdomain when compared to that of the SH domain, allowingfor its low-dimensional approximation. We formulated theprocesses of image formation and reconstruction in the HSHdomain and tested this framework using several combinationof light sources with directions sampled via Monte Carlosampling. Our experimentation showed the effectiveness of1st order HSH basis in representing images under unknowndistant lighting with different number of light sources, incontrast to 1st and 2nd order SH basis. In addition, weconclude that we can use HSH approximation in modelingimages of Lambertian objects using unconstrained model toinfer lighting coefficients, solved using SVD, rather than us-ing time-consuming non-negative least squares optimization.Ongoing efforts are directed towards modeling more realisticsurface BRDF’s where HSH is known to outperform SH interms of approximation accuracy and efficiency.

REFERENCES

[1] K-C. Lee, J. Ho, D. Kriegman, Acquiring linear subspacesfor face recognition under variable lighting, Pattern Analysisand Machine Intelligence, IEEE Transactions on , vol.27, no.5,pp.684-698, May 2005

[2] R. Habel and M. Wimmer, Efficient Irradiance Normal Map-ping, Proceedings of the 2010 ACM SIGGRAPH symposiumon Interactive 3D Graphics and Games, pp. 189-195, 2010.

[3] R. Basri, D. W. Jacobs, Lambertian reflectance and linearsubspaces, Pattern Analysis and Machine Intelligence, IEEETransactions on, vol.25, no.2, pp. 218- 233, Feb 2003

[4] R. Ramamoorthi, A Signal-Processing Framework for Forwardand Inverse Rendering, Ph.D. Dissertation. Stanford University,Stanford, CA, USA, Advisor(s) P. Hanrahan.

[5] P. Gautron, J. Krivanek, S.N. Pattanaik, and K. Bouatouch, ANovel Hemispherical Basis for Accurate and Efficient Render-ing, Proc. Eurographics Symp. Rendering, 2004.

[6] P. P. Sloan, J. Hall, J. Hart, J. Snyder, Clustered principalcomponents for precomputed radiance transfer, ACM Trans.Graph, vol.22, no.3, pp. 382-391, 2003.

[7] J. Koenderink, A. Van Doorn, M. Stavridi, Bidirectional re-flection distribution function expressed in terms of surfacescattering modes, in ECCV,pp. 28-39, 1996.

[8] J. C. Wyant and K. Creath, Basic wavefront aberration theoryfor optical metrology, In Applied optics and Optical Engineer-ing, vol XI , Academic Press, Inc., pp. 27.39, 1992.

[9] O. A. Makhotkin, Analysis of radiative transfer between sur-faces by hemispherical harmonics, Journal of QuantitativeSpectroscopy and Radiative Transfer, Vol. 56, no. 6, 869-879,1996.

[10] P. K. Kythe and M. R. Schaferkotter, Handbook of Computa-tional Methods for Integration, Chapman & Hall/CRC, 2004.

[11] P. Belhumeur and D. Kriegman,What Is the Set of Imagesof an Object Under All Possible Lighting Conditions?, IEEEConf.Computer Vision and Pattern Recognition, pp. 270-277,1996.

[12] S. Sarkar,USF HumanID 3D Face Database, University ofSouth Florida, Tampa, FL, USA.

[13] C. L. Lawson, and R. J. Hanson, Solving Least SquaresProblems, Prentice-Hall, Chapter 23, p. 161, 1974.

[14] R. Bro and S. D. Jong, A Fast Non-negativity-constrainedLeast Squares Algorithm, Journal of Chemometrics,11(5):393401, 1997.

[15] Y. Tikochinsky,An addition theorem for spherical harmonics,Mathematical Proceedings of the Cambridge PhilosophicalSociety, 63, pp 1093-1096, 1967

APPENDIX

A. PROOF OF ADDITION THEOREM FORHEMISPHERICAL HARMONICS

Proof: Let (x′, y′, z′) be the coordinate system whosez-axis is aligned with (θ2, φ2), i.e. a rotated version ofthe system (x, y, z) let u′1 : (γ, ξ) be the spherical angleswhich specify the direction u1 : (θ1, φ1) in (x′, y′, z′).Since a rotation angle βHSH in the hemispherical domainis related to its counterpart βSH in the spherical domain byβSH = arccos(2 cosβHSH−1) [5], and according to [15], arotation of the coordinate system transforms each sphericalharmonic basis function Y mn (θ1, φ1) to a linear combinationof spherical harmonic basis in the new rotated system withinthe same harmonic order n, i.e. Y sn (γ, ξ) with s ∈ [−n, n],with coefficients depending on spherical harmonics rotationmatrices . Therefore, this also holds for hemispherical har-monics basis functions, however the coefficients are changedin accordance to the rotation matrices of the hemisphericalharmonics defined by Gautron et al [5]. Consequently wecan expand Hm

n (θ1, φ1) in the hemispherical harmonics ofthe (γ, ξ) angular variables domain as,

Hmn (θ1, φ1) =

n∑s=−n

amn,sHsn(γ, ξ) (A1)

For the purpose of the proof, we only need the coefficientamn,0 which can be obtained as follows,

amn,0 =

∫Ω+

Hmn (θ1, φ1)H0∗

n (γ, ξ)dΩ+γ,ξ (A2)

Using the definition of hemispherical harmonics (3) withm = 0, we can write Pn(cos γ) as,

Pn(cos γ) =

√2π

2n+ 1H0n(γ, ξ) (A3)

and also it can be expanded in terms of hemisphericalharmonics Hm

n (θ1, φ1) as,

Pn(cos γ) =

n∑m=−n

bn,mHmn (θ1, φ1) (A4)

where the expansion coefficients can be obtained by,

bn,m =

∫Ω+

Pn(cos γ)Hm∗n (θ1, φ1)dΩ+

=

∫Ω+

√2π

2n+ 1H0n(γ, ξ)Hm∗

n (θ1, φ1)dΩ+

Taking the complex conjugate, we will have,

b∗n,m =

√2π

2n+ 1

∫Ω+

Hm∗n (θ1, φ1)H0∗

n (γ, ξ)dΩ+ (A5)

Comparing (A5) with (A2), we will have,

b∗n,m =

√2π

2n+ 1amn,0 (A6)

Now, expand Hmn (θ2, φ2) using hemispherical harmonics in

(γ, ξ) angular variables domain, nothing that the values of(γ, ξ) corresponding to (θ1, φ1) = (θ2, φ2) are (0, 0),

Hmn (θ2, φ2) =

n∑s=−n

amn,sHsn(0, 0) (A7)

= amn,0H0n(0, 0) = amn,0

√2n+ 1

2π

where all terms with s 6= 0 vanish with zero γ and ξ, hence,

amn,0 =

√2π

2n+ 1Hmn (θ2, φ2) (A8)

Substituting (A8) in (A6), and taking the complex conjugate,we will have,

bn,m =2π

2n+ 1Hm∗n (θ2, φ2) (A9)

Now, substitute (A9) in (A4), we will achieve the additionformula as follows,

Pn(cosγ) =2π

2n+ 1

n∑m=−n

Hmn (θ1, φ1)Hm∗

n (θ2, φ2)

(A10)