6
Current Biology 24, 2737–2742, November 17, 2014 ª2014 Elsevier Ltd All rights reserved http://dx.doi.org/10.1016/j.cub.2014.09.074 Report Specular Image Structure Modulates the Perception of Three-Dimensional Shape Scott W.J. Mooney 1, * and Barton L. Anderson 1, * 1 School of Psychology, Griffith Taylor Building (A19), University of Sydney, Sydney, NSW 2006, Australia Summary Retinal images are produced by interactions between a sur- face’s 3D shape, material properties, and surrounding light field. In order to recover the 3D geometry of a surface, the visual system must somehow separate aspects of image structure generated by a surface’s shape from structure generated by its material properties or the light field in which it is embedded. Attributing image structure to the wrong physical source would cause the visual system to interpret changes in one physical property (such as reflectance) as changes in another (such as shape). Many previous studies have shown that the visual system does not conflate image structure generated by specular reflectance with 3D shape [1–6], but they did not assess the physical conditions where it would be computationally most difficult to disentangle these different sources of image structure. Here, we show that varying the specular roughness and curvature of sur- faces embedded in natural light fields can strongly modulate perceived shape. Despite the complexity of these interac- tions, we show how an image’s gradient structure mediates its interpretation as a specular reflection or a change in 3D shape. Our findings provide a coherent explanation of when and why specular reflections impact perceived shape and reveal how the static surface properties, simplified light fields, and experimental methods used in previous studies may explain their inconsistent results. Results and Discussion In order to extract a behaviorally stable representation of 3D surface geometry, the visual system must somehow disen- tangle aspects of image structure produced by shape from other sources of image variation, but the precise physical rela- tionship between 3D shape and image structure varies for different materials. Surfaces characterized by diffuse Lamber- tian reflectance distribute reflected light uniformly in all direc- tions, which generates smooth shading gradients that depend primarily on interactions between the surface geometry and the locations of surrounding light sources. The majority of work on the computation of 3D surface geometry from monoc- ular images has focused on shape cues exhibited by diffuse surfaces [7–16], but most natural surfaces also exhibit some degree of specular reflectance. Failure to distinguish specular image structure from diffuse shading could theoretically cause the visual system to conflate changes in material properties with changes in 3D shape, leading to distortions in the percep- tion of shape and/or material properties of a surface. Under what conditions would the visual system be likely to conflate image structure caused by material properties and shape? A specular surface reflects an image of its environment that is distorted by the surface’s 3D shape. When embedded in a highly simplified light field (such as a single light source), a specular surface will generate only a few isolated specular highlights. Previous studies have found that such reflections have little [1, 2] to no effect [3–5] on monocular perceived shape. However, specular surfaces embedded in realistic light fields generate specular reflections over their entire surface, much like surface shading. Thus, it may be more difficult to disentangle image structure contributed by specular reflec- tions from shading when they are embedded in natural illu- mination fields. The image structure generated by specular reflections also depends on specular surface roughness, which effectively blurs the reflected image of the natural world in ways that can be similar to shading gradients. However, the amount of blur caused by specular roughness also depends on surface curvature. Whereas specular roughness causes im- age blur by scattering incoming light directions, highly curved specular surfaces compress incoming light directions into smaller image regions [17], effectively counteracting the blur- ring effects of specular roughness. More highly curved spec- ular surfaces therefore require a higher amount of specular roughness to generate image gradients that have a similar sharpness to those generated by less-curved surfaces. This suggests that surface curvature should modulate the impact of specular roughness on the formation of image gradients, which in turn should determine when variations in specular roughness are likely to influence perceived 3D shape. An informal test of this hypothesis is presented in the two sequences of surfaces depicted in Figure 1. The top row of Figure 1 depicts a high-relief bumpy sphere with varying reflectance: a mixture of diffuse Lambertian shading and low-roughness specular reflections (left); Lambertian shading and moderate-roughness specular reflections (center); and Lambertian shading only (right). The bottom row depicts a low-relief surface created by compressing the high-relief sphere along the observer’s line of sight, which has the same reflectance and roughness as the surface immediately above it. Side views of these two surfaces are depicted in the right- most panel. There are two striking perceptual effects evident in these two sequences. First, the perceived gloss of the com- pressed sphere (bottom row) appears to decrease much more rapidly than the uncompressed sphere above it. The increase in specular roughness more rapidly blurs the specular struc- ture generated by the low surface curvature than the high-cur- vature surface, causing the more highly curved surface in the top row to appear glossier than the bottom row. This effect is consistent with previous studies that have shown that the vi- sual system uses the sharpness of specular structure to infer surface gloss [18, 19]. The second perceptual consequence of increasing specular roughness involves the perceived 3D shape of the bottom row, which appears to vary as a function of its physical reflectance. All of the shapes within each row of Figure 1 are physically identical, but the surface in the bottom row appears to change shape as a function of specular rough- ness. In contrast, the shape of the more strongly curved sur- face in the top row remains relatively unaffected by changes in roughness, and (comparatively subtle) shape changes only become apparent between the specular and purely *Correspondence: [email protected] (S.W.J.M.), barton. [email protected] (B.L.A.)

Specular Image Structure Modulates the Perception of Three-Dimensional Shape

  • Upload
    bartonl

  • View
    212

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Specular Image Structure Modulates the Perception of Three-Dimensional Shape

Specular Image Structure Mo

Current Biology 24, 2737–2742, November 17, 2014 ª2014 Elsevier Ltd All rights reserved http://dx.doi.org/10.1016/j.cub.2014.09.074

Reportdulates the

Perception of Three-Dimensional Shape

Scott W.J. Mooney1,* and Barton L. Anderson1,*1School of Psychology, Griffith Taylor Building (A19),University of Sydney, Sydney, NSW 2006, Australia

Summary

Retinal images are produced by interactions between a sur-

face’s 3D shape, material properties, and surrounding lightfield. In order to recover the 3D geometry of a surface, the

visual system must somehow separate aspects of imagestructure generated by a surface’s shape from structure

generated by itsmaterial properties or the light field in whichit is embedded. Attributing image structure to the wrong

physical source would cause the visual system to interpretchanges in one physical property (such as reflectance) as

changes in another (such as shape). Many previous studieshave shown that the visual system does not conflate image

structure generated by specular reflectance with 3D shape[1–6], but they did not assess the physical conditions where

it would be computationally most difficult to disentanglethese different sources of image structure. Here, we show

that varying the specular roughness and curvature of sur-faces embedded in natural light fields can stronglymodulate

perceived shape. Despite the complexity of these interac-tions, we show how an image’s gradient structure mediates

its interpretation as a specular reflection or a change in3D shape. Our findings provide a coherent explanation of

when and why specular reflections impact perceived shape

and reveal how the static surface properties, simplified lightfields, and experimental methods used in previous studies

may explain their inconsistent results.

Results and Discussion

In order to extract a behaviorally stable representation of 3Dsurface geometry, the visual system must somehow disen-tangle aspects of image structure produced by shape fromother sources of image variation, but the precise physical rela-tionship between 3D shape and image structure varies fordifferent materials. Surfaces characterized by diffuse Lamber-tian reflectance distribute reflected light uniformly in all direc-tions, which generates smooth shading gradients that dependprimarily on interactions between the surface geometry andthe locations of surrounding light sources. The majority ofwork on the computation of 3D surface geometry frommonoc-ular images has focused on shape cues exhibited by diffusesurfaces [7–16], but most natural surfaces also exhibit somedegree of specular reflectance. Failure to distinguish specularimage structure from diffuse shading could theoretically causethe visual system to conflate changes in material propertieswith changes in 3D shape, leading to distortions in the percep-tion of shape and/or material properties of a surface.

Under what conditions would the visual system be likely toconflate image structure caused by material properties and

*Correspondence: [email protected] (S.W.J.M.), barton.

[email protected] (B.L.A.)

shape? A specular surface reflects an image of its environmentthat is distorted by the surface’s 3D shape.When embedded ina highly simplified light field (such as a single light source), aspecular surface will generate only a few isolated specularhighlights. Previous studies have found that such reflectionshave little [1, 2] to no effect [3–5] on monocular perceivedshape. However, specular surfaces embedded in realistic lightfields generate specular reflections over their entire surface,much like surface shading. Thus, it may be more difficult todisentangle image structure contributed by specular reflec-tions from shading when they are embedded in natural illu-mination fields. The image structure generated by specularreflections also depends on specular surface roughness,which effectively blurs the reflected image of the natural worldin ways that can be similar to shading gradients. However, theamount of blur caused by specular roughness also dependson surface curvature.Whereas specular roughness causes im-age blur by scattering incoming light directions, highly curvedspecular surfaces compress incoming light directions intosmaller image regions [17], effectively counteracting the blur-ring effects of specular roughness. More highly curved spec-ular surfaces therefore require a higher amount of specularroughness to generate image gradients that have a similarsharpness to those generated by less-curved surfaces. Thissuggests that surface curvature should modulate the impactof specular roughness on the formation of image gradients,which in turn should determine when variations in specularroughness are likely to influence perceived 3D shape.An informal test of this hypothesis is presented in the two

sequences of surfaces depicted in Figure 1. The top row ofFigure 1 depicts a high-relief bumpy sphere with varyingreflectance: a mixture of diffuse Lambertian shading andlow-roughness specular reflections (left); Lambertian shadingand moderate-roughness specular reflections (center); andLambertian shading only (right). The bottom row depicts alow-relief surface created by compressing the high-reliefsphere along the observer’s line of sight, which has the samereflectance and roughness as the surface immediately aboveit. Side views of these two surfaces are depicted in the right-most panel. There are two striking perceptual effects evidentin these two sequences. First, the perceived gloss of the com-pressed sphere (bottom row) appears to decrease much morerapidly than the uncompressed sphere above it. The increasein specular roughness more rapidly blurs the specular struc-ture generated by the low surface curvature than the high-cur-vature surface, causing the more highly curved surface in thetop row to appear glossier than the bottom row. This effectis consistent with previous studies that have shown that the vi-sual system uses the sharpness of specular structure to infersurface gloss [18, 19]. The second perceptual consequenceof increasing specular roughness involves the perceived 3Dshape of the bottom row, which appears to vary as a functionof its physical reflectance. All of the shapes within each row ofFigure 1 are physically identical, but the surface in the bottomrow appears to change shape as a function of specular rough-ness. In contrast, the shape of the more strongly curved sur-face in the top row remains relatively unaffected by changesin roughness, and (comparatively subtle) shape changesonly become apparent between the specular and purely

Page 2: Specular Image Structure Modulates the Perception of Three-Dimensional Shape

Figure 1. Two Distinct Physical Shapes with Varying Reflectance Properties

The surface in the top row is a bumpy sphere with high relief and high overall curvature. This sphere has been compressed in the bottom row, as shown in the

side view in the right panel. The surfaces in the left panel are depicted with a mixture of Lambertian shading and low-roughness specular reflections (left),

shading andmoderate-roughness specular reflections (center), and shading only (right). Perceived 3D shape changes dramatically as a function of specular

reflectance in the bottom row. Apparent changes in both material and shape for the sphere in the top row are much smaller by comparison. See also

Figure S1 (available online) for examples of our experimental stimuli rendered in other natural light fields, which demonstrates that the effects reported

are not isolated to the particular natural light field used.

Current Biology Vol 24 No 222738

Lambertian materials. Here, we focus on how interactions be-tween the curvature and reflectance properties of surfacesmodulate their perceived shape.

In order to quantitatively assess the interaction betweenspecular reflectance, surface curvature, and perceived shape,we created an array of deformed planar surfaces by addingpseudorandom ‘‘dents’’ to a plane (Figure 2). The contours sur-rounding each dent therefore correspond to a discontinuity inthe derivative in surface orientation that adjoins the dent to theperturbed plane (similar to a level cut). The local curvature ofeach dent was varied bymodulating their relief height. The sur-faces were embedded in a light field captured from a naturaloutdoor scene, and the amount of specular roughness andsurface curvature were parametrically varied by modulating aspecular distribution parameter and 3D relief, respectively(see the Experimental Procedures). Variations in global con-trast produced by these manipulations were reduced bynormalizing the intensity range of each image. The original pre-normalization renders are depicted in Figure S2. Each row inFigure 2 depicts surfaces with identical surface geometry,which increases in global relief height (and hence dent curva-ture) from top to bottom. The first five columns depict surfacereflectance functions with both shading and specular reflec-tions. Specular roughness increases from left to right, whichvisibly increases the amount of reflection blur. The columnon the far right depicts surfaces with Lambertian shadingonly. A schematic of each reflectance function is depictedabove each column; the dark gray arrow represents anincoming light ray and the light gray distribution representsthe proportion of light reflected in each direction by the sur-face. The circular parts of these distributions depict diffuse(Lambertian) reflectance, whereas the direction-specific lobes(bumps on the right-hand side of the figures) depict specularreflectance. This lobe becomes shorter and wider as specularroughness is increased. Note that the perceived surface relief

appears to become progressively flatter as specular rough-ness is increased and that the rate of this apparent flatteningdiffers for each level of physical surface relief. The reducedcurvature of low-relief surfaces (upper rows) causes the spec-ular image gradients to blur more rapidly as a function of spec-ular roughness than surfaces with greater curvature (lowerrows), which modulates where the greatest differences in per-ceived shape are experienced. This effect is robust to changesin the particular natural light field used to render the surfaces(see Figure S1 for examples of the lowest relief surfacerendered in other light fields).We designed two sets of experiments to quantify how

perceived shape is modulated by specular roughness and sur-face curvature. In order to simplify our shapemeasurements toa 1D cross-section, we chose a surface region where the sur-face curvature varied almost exclusively in one direction in theimage (here, horizontally). In experiment 1a, observers wereshown a selection of six surfaces (outlined in black in Figure 2)and adjusted gauge figure probes to match the perceived sur-face normals at 20 points that traversed a surface ridge (seethe Experimental Procedures and the white line in Figure 3C).The curvature of this ridge varied as a function of parametricsurface relief. Numerous studies have found the gauge figuretask to be a reliable and intuitive measure of perceived shape[2, 5, 6, 20, 21]; a screenshot from the task is depicted inFigure 3B. Gauge figure settings were then integrated toreconstruct profiles of perceived 3D shape along the sampled1D cross-section (see the Supplemental Information). Theaverage reconstructed shape profiles are shown in Figure 3A.Each graph plots perceived relief height as a function of probeposition. The top panel depicts the profiles for the lower reliefsurface, and the bottom panel depicts the profiles for thehigher relief surface. The light gray dotted, dark gray dotted,and black solid curves depict the profiles of the low-roughnessspecular, moderate-roughness specular, and shading-only

Page 3: Specular Image Structure Modulates the Perception of Three-Dimensional Shape

Figure 2. Stimuli Used in Our Experiments

Stimuli for the experiments presented. Each row depicts surfaces with a different level of physical surface relief, increasing from top to bottom. The cross-

section schematics on the left demonstrate how the curvature of the central ridge changes as surface relief increases. The first five columns show surfaces

with both Lambertian shading and specular reflections, with specular roughness increasing from left to right. The sixth column shows surfaces with shading

only. The schematic above each column depicts these reflectance functions, comprising a circular diffuse Lambertian distribution and a direction-specific

specular lobe that widens as specular roughness is increased. In order to control for the effects of global image contrast, the intensity range of each image

has been normalized. Note the differences in perceived 3D shape that occur within each row as a function of surface reflectance. The six surfaces used in

experiment 1a are outlined in black. See also Figure S2 for globally tone-mapped versions of these images (used in experiments 1b and 2b).

Specular Structure Modulates Perception of Shape2739

materials, respectively. The black dotted curves depict theveridical profile of each surface. Within-subjects ANOVAs re-vealed quantitative distortions in perceived shape producedby interactions between specular blur and surface curvature.At lower relief, a significant difference in perceived shapewas only observed between the low-roughness and moder-ate-roughness specular surfaces (p < 0.005) and betweenthe low-roughness specular and shading-only surfaces (p <0.001). The reduced curvature of this ridge increases theimpact of specular roughness on image blur and causes mod-erate-roughness reflections to closely resemble the gradientsgenerated by the shading-only surface. At higher relief, signif-icant quantitative shape differences were observed for eachpair of reflectance types (all p < 0.001). In experiment 1b, thesame observers participated in an identical task for the corre-sponding surfaces without normalized image contrast (Fig-ure S2). The pattern of results was identical (see Figure S3),indicating that changes in global image contrast alone werenot responsible for the differences between materials.

The preceding experiment reveals that quantitative changesin perceived shape can be elicited by varying specular reflec-tance properties and that the magnitude of this effect varies asa function of surface curvature. More specular roughness isrequired to abolish the formation of specular image gradientsthat run along directions of high curvature than low curvaturedue to the compression of specular image structure generatedby high-curvature surface regions [17]. If these differences inimage structure are responsible for differences in perceived

shape, then it implies that a different amount of specularroughness is needed to induce changes in perceived shapefor surfaces with different amounts of surface curvature. Spe-cifically, this analysis predicts that the amount of specularroughness required to induce a change in the perceived 3Dshape of a fixed surface geometry should increase monotoni-cally as a function of surface curvature (relief). In the context ofour stimuli depicted in Figure 2, this analysis predicts that thepair of adjacent surfaces that exhibits the largest difference inperceived shape should progressively shift toward greateramounts of specular roughness (i.e., toward the right) as sur-face relief is increased (i.e., from the top row to the bottomrow). To assess this prediction, observers in experiment 2awere shown two pairs of specular surfaces from Figure 2 andindicated which pair appeared most different in 3D shape.Each pair of surfaces had physically identical surface reliefand differed only by one step in parametric specular rough-ness (see the Experimental Procedures). The same task wasused in experiment 2b using the stimuli without normalizedimage contrast (Figure S2). The results of experiment 2a aredepicted in the left column of Figure 4C. The vertical axis rep-resents the percentage of trials in which each pair of surfaceswas selected as appearing most different in shape. Each rowshows a different surface relief level, and each shade depictsa particular step in roughness. The data reveal that the changein specular roughness that produces the largest change inperceived shape depends on the level of surface relief. A smallamount of roughness has the largest effect on perceived

Page 4: Specular Image Structure Modulates the Perception of Three-Dimensional Shape

Figure 3. Average Shape Profiles Reconstructed

from Observer Measurements of Surface Orienta-

tion in Experiment 1

(A) Reconstructed shape profiles from experiment

1a. The top panel depicts surfaces from the lower

relief level, and the bottom panel depicts surfaces

from the higher relief level. The horizontal axis rep-

resents probe location, and the vertical axis repre-

sents the height of the contour in the same units of

distance. The light gray dotted, dark gray dotted,

and black solid lines depict the shape profiles for

the low-roughness specular, moderate-rough-

ness specular, and shading-only reflectance func-

tions, respectively. Error bars represent 61 SEM.

The black dotted curves represent the actual

profiles of the two physical shapes. If perceived

shape was not affected by changes in surface

reflectance, we would expect the profiles within

each panel to be identical. Instead, we found large

differences in perceived 3D shape as a function of

surface reflectance and found that the precise na-

ture of this effect also depends on surface relief.

(B) Screenshot from the gauge figure task.

Observers adjust the orientation of the probe until

its orientation matches that of the underlying sur-

face. The probe here has been changed to white

and increased in size by 100% for visibility.

(C) Sample image demonstrating the horizontal

contour where perceived surface orientation was

measured (the white line).

See also Figure S3.

Current Biology Vol 24 No 222740

shape at the lower relief levels, but this peak shifts towardlarger amounts of roughness as surface relief increases. Thesame pattern of results was observed in experiment 2b, whichindicates that differences in image contrast cannot account forthese effects (see Figure S4).

The preceding results reveal that the effects of specularroughness on perceived shape vary systematically as a func-tion of surface curvature. This can be experienced directly inFigure 2. The largest differences in perceived shape withineach row (i.e., as a function of specular roughness) occurwhen the central ridge transitions from appearing sharpand protrusive to smoother and flatter, which appears tocorrespond to where specular roughness induces the largestdifferences in the gradient structure across this ridge. If thishypothesis is correct, it suggests that it should be possibleto predict the order of observers’ shape difference judgmentsin experiment 2a by computing the difference between theluminance gradients in each pair of images tested; the imagepair with the largest difference in gradient structure shouldinduce the largest change in 3D shape. To test this hypothesis,we approximated the first derivative of image intensity alongthe contours sampled in experiment 1a (see the SupplementalExperimental Procedures). In order to eliminate pixel noise anddiminish the influence of the high-frequency edges of specularreflections generated at low degrees of specular roughness, all20 specular surfaces were first filtered using a small amount ofGaussian blur. This small amount of blur does not impact howshape appears to change within each row (compare Figures 2and 4A). The filter had a size and SD of 2.3% and 1.6% of eachstimulus’s width, respectively (see the Supplemental Informa-tion). These blurred images are depicted in Figure 4A, and thecorresponding extracted image gradients are shown in Fig-ure 4B. The opposing effects of surface curvature and specularroughness on gradient structure are clearly visible in this array:increasing specular roughness (i.e., moving to the right withineach row) stretches the gradients outward by blurring the

image, but the rate of this effect is lessened as surface relief in-creases (i.e., moving down the rows). Note that the largestchanges in perceived shape in Figure 4A correspond to thepairs of derivative functions that exhibit the largest differ-ences. This can be shown by calculating the difference be-tween each pair of derivative functions and then comparingthe maximum amplitudes of the resulting difference functionsto the data fromexperiment 2a. These amplitudes are depictedin the right column of Figure 4C and closely match the orderingof perceived differences in shape as a function of changingspecular roughness for each surface relief level, as well asthe overall pattern of results.The stimuli and experiments presented herein demonstrate

that changes in the reflectance properties of specular surfacescan impact our perception of 3D shape. Previous work has es-tablished a set of image cues that predict when variations inshape are likely to modulate perceived gloss [18, 19], but theeffects of surface reflectance on perceived shape have previ-ously been found to be relatively minor or nonexistent [1–6].We find that the modulatory effects of specular reflectanceon perceived shape occur for surfaces embedded in naturallight fields that contain some degree of specular roughness.These conditions make it particularly difficult to disentanglethe contributions of specular reflections from shading gradi-ents. Natural light fields cover specular surfaces with imagesof the light field, and the image blur induced by specularroughness ‘‘blends’’ the image gradients generated by spec-ular reflections into image gradients generated by diffuse sur-face shading. The modulatory effects of specular reflectanceon perceived shape can be understood as a computationalfailure in separating these distinct sources of image structure.However, the modulatory effect of specular roughness on im-age blur—and hence shape—also depends on the local curva-ture of surfaces. Highly curved surface regions generate morecompressed specular reflections and therefore require greatervalues of specular roughness before the image gradients they

Page 5: Specular Image Structure Modulates the Perception of Three-Dimensional Shape

Figure 4. Sample Image Gradient Derivatives and Results from Experiment 2a

(A) Stimuli used in experiments 1a and 2a with a small amount of Gaussian blur applied in order to eliminate image noise and high-frequency specular

gradient structure. Note that the pairs of adjacent surfaces that appear most different in shape are essentially unaffected, even though the extremely

high-frequency specular reflections in the bottom-left quadrant of the array are eliminated (compare to Figure 2).

(B) Discrete approximations of image gradients from each corresponding image in (A), sampled along the horizontal contour used to measure perceived

shape in experiment 1a (the white line in Figure 3C). The steps in specular roughness that have the largest effect on perceived shape within each row

also have the largest impact on this gradient structure.

(C) Results from experiment 2a (left column) compared to the amplitudes of difference functions calculated for pair of adjacent derivative plots in (B; right

column). Each row represents a different level of surface relief, and each shade represents a different step in specular roughness. In the left column, the

vertical axis depicts the percentage of trials in which each pair was selected as appearing most different in shape. Error bars represent61 SEM. In the right

column, the vertical axis denotes maximum peak-to-trough amplitude in units of luminance change per pixel (see the Supplemental Information). These

amplitudes closely predict the ordering of perceived differences in shape as a function-of-roughness step for each level of relief.

See also Figure S4 for the raw calculated difference functions and the results from experiment 2b.

Specular Structure Modulates Perception of Shape2741

generate converge with the gradients generated by actual sur-face shading.

A growing body of work has revealed that our visual percep-tion of a variety of surface properties can be affected by phys-ically independent sources of image structure [18, 19, 22–27].Our impressions of shape,material, and illumination aremodu-lated by the particular way that image structure is affected bythe interaction between these different sources of image vari-ability. The phenomena and experiments reported abovereveal that the perceptual conflation of specular reflectancewith 3D shape can be understood by considering the

generative constraints that cause specular reflections tocreate image structure similar to that generated by surfaceshading. Our results offer a general approach for using gener-ative models to predict when one source of image structure ismost likely to be perceptually conflated with a physically inde-pendent source of structure. This analysis begins by consid-ering the kind of image structure that is typically generatedby one source (such as surface shading) and proceedsby considering the generative conditions that could causea distinct physical source (such as specular reflections) toproduce similar image structure. Whereas a large body of

Page 6: Specular Image Structure Modulates the Perception of Three-Dimensional Shape

Current Biology Vol 24 No 222742

empirical and theoretical work has been devoted to under-standing perceptual constancies—i.e., the ability of the visualsystem to effectively discount variations in the input to recoveran invariant property of the world—the approach exploitedpresently provides a method for predicting constancy failures,which in turn can provide insight into the image properties thatare used by the visual system to derive scene attributes.

Experimental Procedures

Surfaces were created in Blender 2.65 by deforming the shape of a 20 cm

square plane according to a distorted noise texture. The maximum relief

height of this deformed surface was scaled differently for each relief level:

0.1 cm; 0.2 cm; 0.4 cm; or 0.8 cm. Each surface was rendered with six reflec-

tance functions: purely diffuse Lambertian light gray reflectance and five

mixed reflectance functions combining this diffuse reflectance with a spec-

ular component varying in a specular roughness parameter. The surfaces

were rendered in a natural light field (‘‘Meadow Trail’’) from a perspective

viewpoint 60 cm above the surface. The resulting 32-bit images were simul-

taneously tone mapped into greyscale 8-bit images (Figure S2) and then

individually had their intensity range normalized to reduce differences in

global image contrast (Figure 2). In experiments 1a and 1b, observers

adjusted gauge figure orientation probes at 20 points along an image con-

tour for six images from Figures 2 and S2, respectively, with three repeats

each. These orientation settings were averaged and integrated to recon-

struct profiles of perceived shape for each observer, which were compared

using within-subject ANOVAs. In order to create the mean profiles depicted

in Figures 3A and S3, between-subject variation in overall perceived relief

height was discounted by normalizing relief height scale across observers.

In experiments 2a and 2b, observers were shown two pairs of specular sur-

faces from Figures 2 and S2, respectively, and selected the pair that ap-

peared to show the largest change in 3D shape. The surfaces within each

pair had identical relief and differed by one step in specular blur. Methods

are described in more detail in the Supplemental Experimental Procedures.

Observers were recruited with the approval of the University of Sydney’s

Human Research Ethics Committee.

Supplemental Information

Supplemental Information includes Supplemental Experimental Procedures

and four figures and can be foundwith this article online at http://dx.doi.org/

10.1016/j.cub.2014.09.074.

Acknowledgments

This project was funded by an Australian Research Council grant awarded

to B.L.A.

Received: July 29, 2014

Revised: September 11, 2014

Accepted: September 29, 2014

Published: November 6, 2014

References

1. Aloimonos, J. (1988). Shape from texture. Biol. Cybern. 58, 345–360.

2. Erens, R.G., Kappers, A.M., and Koenderink, J.J. (1993). Perception of

local shape from shading. Percept. Psychophys. 54, 145–156.

3. Horn, B.K., and Brooks, M.J. (1989). Shape from Shading (Cambridge:

MIT Press).

4. Kleffner, D.A., and Ramachandran, V.S. (1992). On the perception of

shape from shading. Percept. Psychophys. 52, 18–36.

5. Langer, M.S., and Bulthoff, H.H. (2000). Depth discrimination from

shading under diffuse lighting. Perception 29, 649–660.

6. Li, A., and Zaidi, Q. (2000). Perception of three-dimensional shape

from texture is based on patterns of oriented energy. Vision Res. 40,

217–242.

7. Liu, B., and Todd, J.T. (2004). Perceptual biases in the interpretation of

3D shape from shading. Vision Res. 44, 2135–2145.

8. Mamassian, P., and Kersten, D. (1996). Illumination, shading and the

perception of local orientation. Vision Res. 36, 2351–2367.

9. Pentland, A. (1989). Shape information from shading: a theory about hu-

man perception. Spat. Vis. 4, 165–182.

10. van Doorn, A.J., Koenderink, J.J., and Wagemans, J. (2011). Light fields

and shape from shading. J. Vis. 11, 1–21.

11. Faisman, A., and Langer, M.S. (2013). Qualitative shape from shading,

highlights, and mirror reflections. J. Vis. 13, 1–16.

12. Norman, J.F., Todd, J.T., and Phillips, F. (1995). The perception of sur-

face orientation from multiple sources of optical information. Percept.

Psychophys. 57, 629–636.

13. Todd, J.T., and Mingolla, E. (1983). Perception of surface curvature and

direction of illumination from patterns of shading. J. Exp. Psychol. Hum.

Percept. Perform. 9, 583–595.

14. Mingolla, E., and Todd, J.T. (1986). Perception of solid shape from

shading. Biol. Cybern. 53, 137–151.

15. Nefs, H.T., Koenderink, J.J., and Kappers, A.M. (2006). Shape-from-

shading for matte and glossy objects. Acta Psychol. (Amst.) 121,

297–316.

16. Fleming, R.W., Torralba, A., and Adelson, E.H. (2004). Specular reflec-

tions and the perception of shape. J. Vis. 4, 798–820.

17. Fleming, R.W., Torralba, A., and Adelson, E.H. (2009). Shape from

sheen. MIT Tech Report MIT-CSAIL-TR-2009-051.

18. Marlow, P.J., Kim, J., and Anderson, B.L. (2012). The perception and

misperception of specular surface reflectance. Curr. Biol. 22, 1909–

1913.

19. Marlow, P.J., and Anderson, B.L. (2013). Generative constraints on im-

age cues for perceived gloss. J. Vis. 13, 1–23.

20. Koenderink, J.J., van Doorn, A.J., and Kappers, A.M. (1992). Surface

perception in pictures. Percept. Psychophys. 52, 487–496.

21. Koenderink, J.J., van Doorn, A.J., Kappers, A.M., and Todd, J.T.

(2001). Ambiguity and the ‘mental eye’ in pictorial relief. Perception

30, 431–448.

22. Belhumeur, P.N., Kriegman, D.J., and Yuille, A.L. (1999). The bas-relief

ambiguity. Int. J. Comput. Vis. 35, 33–44.

23. Fleming, R.W. (2014). Visual perception of materials and their proper-

ties. Vision Res. 94, 62–75.

24. Fleming, R.W., Dror, R.O., and Adelson, E.H. (2003). Real-world illu-

mination and the perception of surface reflectance properties. J. Vis.

3, 347–368.

25. Ho, Y.X., Landy, M.S., and Maloney, L.T. (2008). Conjoint measurement

of gloss and surface texture. Psychol. Sci. 19, 196–204.

26. Olkkonen, M., and Brainard, D.H. (2010). Perceived glossiness and light-

ness under real-world illumination. J. Vis. 10, 5.

27. Olkkonen, M., and Brainard, D.H. (2011). Joint effects of illumination ge-

ometry and object shape in the perception of surface reflectance.

Iperception 2, 1014–1034.