1 IntroductionThe question how the brain groups together fragmentary visual information fromparts of the same surface that are separated by occluders has been raised many yearsago by the Gestalt psychologists (Koffka 1935; Kanizsa 1950, 1979), and has recentlygained renewed attention (Kellman and Shipley 1991; Peterhans and von der Heydt1991; Nakayama et al 1995). Occlusion occurs in virtually every real-world scene, andtherefore surface completion is a fundamental visual process. Michotte (1963) coinedthe term amodal completion to describe the process by which two disconnected regionsin the image are seen to complete behind the occluder' and form a single surface(figure 1a). The term `amodal' refers to the fact that observers do not actually see thecompleted' part of the imageöwhich is behind the occluderöthey merely `know it isthere' (see Koffka 1935, page 180). This is in contrast with modal completionöa situationwhere the completed surface is in front, and is therefore experienced visually, givingrise to illusory contours in the visually (`modally') completed parts of the boundingcontour (figure 2a).

An important observation about surface completion is that it can take place in theabsence of prior knowledge, or familiarity with the shapes of the completed surfaces(Nakayama et al 1995). This is true both for amodal and for modal completion, ascan be seen in figures 1a and 2a, respectively. This marks surface completion as anintermediate-level visual process; on the one hand, it is largely independent of high-level visual processes such as object recognition and scene perception (but see Peterson1994, 1999; Peterson and Gibson 1994). As we shall see, surface completion is alsohighly sensitive to subtle image manipulations and, in this sense, highly stimulus-driven

The role of junctions in surface completion and contourmatchingÀ

Perception, 2001, volume 30, pages 339 ^ 366

Nava RubinCenter for Neural Science, New York University, 4 Washington Place, New York, NY 10003, USA;e-mail: nava@cns.nyu.eduReceived 16 January 1999, in revised form 8 July 2000

Abstract. It has been suggested that contour junctions may be used as cues for occlusion.Ecologically, T-junctions and L-junctions are concurrent with situations of occlusion: they arisewhen the bounding contour of the occluding surface intersects with that of the occluded surface.However, there are other image properties that can be used as cues for occlusion. Here therole of junctions is directly compared with other occlusion cuesöspecifically, relatability andsurface-similarityöin the emergence of amodal completion and illusory contour perception.Stimuli have been constructed that differ only in the junction structure, with the other occlusioncues kept unchanged. L-junctions and T-junctions were eliminated from the image or manipu-lated so as to be locally inconsistent with the (still valid) global occlusion interpretation.Although the other occlusion cues of relatability and surface similarity still existed in the image,subjects reported not perceiving illusory contours or amodal completion in junction-manipulatedimages. Junction manipulation also affected the perceived stereoscopic depth and motion of imageregions, depending on whether they were perceived to amodally complete with a disjoint regionin the image. These results are interpreted in terms of the role of junctions in the processes ofsurface completion and contour matching. It is proposed that junctions, being a local cue forocclusion, are used to launch completion processes. Other, more global occlusion cues, such asrelatability, play a part at a later stage, once completion processes have been launched.

DOI:10.1068/p3173

À Parts of this work were presented at the 1997 meeting of the Association for Research in Visionand Ophthalmology (N Rubin 1997).

(although it can sometimes be affected by attention and by learning; see Coren et al1987; Rock 1987; Wallach and Slaughter 1988; N Rubin et al 1997). On the other hand,surface completion is richer and more complex than early visual processes such ascontrast and edge detection. A principal manifestation of this is the fact that surfacecompletion involves global computations: information from disjoint parts of the imageneeds to be integrated for completion to occur. These surface fragments can be quitedistant: occluding surfaces, and even illusory contours, can span more than 10 degvisual angle (Ringach and Shapley 1996). Therefore, the kind of models often invokedfor early visual processes, which consist of a set of local filters that operate independ-ently across the image, cannot be used to explain perceptual completion.

Several physiological studies report that early visual cortical areas can show responsesto perceptually completed contours (von der Heydt et al 1984; Peterhans and von derHeydt 1989; Grosof et al 1993; Hirsch et al 1995; Kapadia et al 1995; Sheth et al 1996;Bakin et al 1998; Mendola et al 1999; Sugita 1999). This is not in conflict with the noteabove about the inadequacy of certain models of early visual cortical cells. Although cellsin early visual areas are known to have small receptive fields, these areas may never-theless support long-range interactions. One possibility is to rely on the long-rangelateral connections which have been reported for early cortical areas (Gilbert andWiesel 1983; Lund 1988; Gilbert et al 1996), and construct units which pool informa-tion from the disjoint fragments (see, eg, Grossberg and Mingolla 1985; Heitger andvon der Heydt 1993). But there is an alternative to this idea: that visual information ismediated across large gaps by a cascade of shorter-range lateral connections betweensmall receptive-field cells. This approach may be better, because it does not require toposit specialized units which are dedicated to contour completion (by individually inte-grating information from widely separated parts of the visual field). Instead, it departsonly minimally from the commonly held view of early visual cortex, by noting thateven short-range connections between small receptive-field cells can give rise to long-range interactions, ie these cells should not be viewed as `independent'. It is thereforeof interest to investigate the role that local image features, which may be coded byindividual cells or small groups of cells, play in completion processes. In this paperresults are presented that support the hypothesis that T-junctions and L-junctions serveas such local features: they are used as `local evidence' for occlusion, and launchcompletion processes, which subsequently may give rise to long-range interactions.

When the bounding contour of an occluding surface intersects with that of anoccluded surface, a two-dimensional (2-D) discontinuity in the image, called a junction,is produced.(1) In the case when the two surfaces have luminances (or colors) that aredifferent from the background and from each other, a T-junction is formed (figure 1b).If the occluding surface has the same luminance (and color) as the background, oneof the arms of the `T' becomes invisible, which results in an L-junction(2) (figure 2b).Because the occurrence of junctions in the image is correlated with occlusion situations,(3)

(1) For a discussion of an interesting exception to this rule, which involves situations when twoobjects are in contact, see Tse and Albert (1998). This paper is concerned only with the genericcase of `overlay' occlusion: that which is created as a result of projecting two objects which areseparated in 3-D onto a 2-D retina.(2) A third case is when the occluding surface is transparent; when a transparent occluding surfaceintersects the bounding contour of an occluded surface, an X-junction is formed. This case is notconsidered in this paper; see Kersten (1991) for a demonstration similar in spirit to those presentedhere about the role of X-junctions in the perception of transparency. See also Metelli (1974);Adelson and Anandan (1990); Anderson (1997).(3) A systematic assessment of the degree of correlation between the occurrence of junctions innatural scenes and the presence of occlusion in the corresponding regions has not yet beenperformed. Such an analysis is obviously a considerable undertaking, which involves both therefinement of automated methods to detect local image regions that should be classified as a junction

340 N Rubin

they can be used as powerful cues to signal occlusion and lead to modal and amodalcompletion. Indeed, many authors have discussed the close relationship between thestructure of 2-D junctions and the underlying surface geometry (Ratoosh 1949; Chapanisand McCleary 1953; Dinnerstein and Wertheimer 1957; Guzman 1969; Metelli 1974;Malik 1987; Shipley and Kellman 1990; Anderson and Julesz 1995; Kapadia et al 1995;Kumaran et al 1996; Anderson 1997; see also Benary 1924). But there are other imageproperties that are also correlated with occlusion situations. Two additional occlusioncues are shown in figure 1. First, observe that the position and orientation of the frag-ments of the bounding contour of the occluded shape on the two sides obey what Kellmanand Shipley (1991) termed `relatability'öroughly speaking, this means that they can be

(3) (continued)and a manual labor-intensive labeling of all the occlusion-related junctions in the scene. Therequired analysis is further complicated by the fact that the relevant edges and junctions may occurat different levels of blur, owing to variations in illumination, focal depth, and other factors (seefigure 18). Nevertheless, for the case of T-junctions it is probably reasonable to assume that a highdegree of correlation will be found. What the outcome of a natural-scene analysis will be for thecase of L-junctions and modal completion (illusory contours) is, on the other hand, less obvious,since cusps in the bounding contours of surfaces due to the intrinsic shape of the surface are notentirely uncommon; this is true not only in the modern, urban visual world (eg the corners oftables, buildings, books) but also in the wild (eg the tips of leaves).

(a) (b) (c)

Figure 1. (a) The perceptual completion of an occluded surface from its retinal-image fragmentsis a basic visual phenomenon; it does not require familiarity with the shape of the completed surface.(b) Where the bounding contour of an occluding surface intersects the bounding contour of anoccluded surface, a T-junction is formed. The junction can be used as a local occlusion cue. (Notethat, in spite of its name, aT-junction will not be 908 in general.) Other cues for occlusion are surfacesimilarity (in this case, the luminance) and bounding-contour relatability [shown in (c)].

(a) (b)

Figure 2. (a) When the occluding surface has the same color and luminance as the background,parts of its bounding contour are not given as luminance-defined edges in the image and thesurface is therefore said to be `illusory'. Its existence is signaled by the visible fragments of theoccluded surfaces (`the inducers'). (b) Where the bounding contour of the illusory occludingsurface intersects the bounding contour of an occluded surface, an L-junction is formed. Thisjunction can also be used as a local occlusion cue (again note that it is not 908 in general).

The role of junctions in surface completion and contour matching 341

connected with a smooth contour without inflection points (figure 1c; see also Singh andHoffman 1998). Second, the fact that disjoint image regions which belong to the samesurface tend to have similar properties such as color or texture may also be used as acue for occlusion. Here, this type of cue is termed surface similarity. These propertiesmay therefore be used as cues for occlusion: detection of relatability of disjoint edgesin the image may indicate that these edges are parts of the bounding contour of aunitary surface, which are separated by another, occluding, surface. Detection of sur-face similarity of disjoint image regions may similarly indicate that these regionsbelong to the same surface.

But junctions are distinguished from relatability and surface similarity in animportant aspect. Junctions are a local property of the imageöthey arise in spatiallylocalized parts of the image and therefore can be detected by operators of relativelysmall spatial extent, or `receptive fields' in neural terms. In contrast, relatability andsurface similarity require global computations, of relating information from spatiallydistant parts of the image. It is therefore reasonable to hypothesize that junctionsplay a privileged role in the emergence of perceptual completion. Here, evidence isprovided for this hypothesis. Stimuli were used in which the junction structure wasmanipulated while leaving relatability and surface similarity unchanged. The manipu-lation of T-junctions and L-junctions can have a decisive effect on perceptual depthstratification of surfaces and on the emergence of illusory contours, respectively.It is proposed that the fact that junctions are local cues for occlusion makes themparticularly suitable for launching completion processes. Ultimately, the perceptualinterpretation given to an image will depend on other occlusion cues as well; but theanalysis of local junction structure can be used to bootstrap the complex process offinding global structure in an image which is initially represented by a set of local units.

2 Amodal completion: T-junctions versus global occlusion cuesIn general, different cues for occlusionöand, specifically, the three types of cuesconsidered here: T-junctions, relatability, and surface similarityöwill tend to co-occur,and presumably enhance each other's effect. But it is possible to construct stimuli thatisolate the effect of T-junctions alone, by manipulating the image such that only thejunction structure is altered, while leaving other occlusion cues unchanged. Considerthe stimuli shown in figures 3a and 3b. In each of the figures, there are two E-shapedoccluders that separate three black regions. We can now test whether the threeblack regions are perceptually completed into a single surface by separating the twooccluders from the two outer black patches using stereo or motion, and asking whetherthe central black patchötermed here the test patchö is seen to have the depth value,or motion, of the occluders, or of the flanking black patches. The stimuli shown infigure 3 therefore overcome the problem that the parts of surfaces which are completedbehind occluders are not experienced perceptually. While it is not possible to askdirect, perceptual questions about the `amodally completed' parts of surfaces, in stimulilike those shown in figure 3 (as well as others shown below) it is possible to ask suchquestions about the test patch, and the answers will depend on whether or not amodalcompletion of the occluded parts took place.

The subtle difference between the stimuli in figures 3a and 3b is what allows us toassess the role of T-junctions in the perceptual completion of the three black regionsinto a single surface. In figure 3a, a T-junction is formed at the intersection betweenthe bounding contour of the occluders and the test patch (ie, it represents the generic,or control case). In figure 3b, in contrast, the shape of the occluders was changedslightly, so that the junctions formed at the intersections of the bounding contours areno longer consistent with the generic situation of occlusion of the gray surface over

342 N Rubin

the black surface.(4) Note, that the black regions are unchanged in figures 3a and 3böit isonly the shape of the occluders that has been manipulated. Therefore, the relatabilityand the surface similarity cues for occlusion are identical in the two stimuli. Whatevercontribution these cues have to the emergence of surface completion, it cannot beaffected by the manipulation done to produce the stimulus in figure 3b.

2.1 Experiment 1: T-junctions and stereoscopic depthIn the first experiment, stereoscopic depth was manipulated in such a way that the depthof the test patch was ambiguous, and depended on whether an occlusion interpretationwas given to the scene, and on the associated resolution of the border-ownership ambi-guity (see below). Observers were asked about the perceived depth of the central test patch.

2.1.1 Stimuli. The stimuli from figures 3a and 3b were made into two stereograms bygiving the occluders crossed disparity (`in front') with respect to the background frame.The horizontal black stripe was left in the same position with respect to the back-ground frame in the images of the two eyes, ie it had zero stereoscopic disparity. Tworeference points with zero disparity were added to the display above and below thetest patch to facilitate subjects' decision about stereoscopic depth. The control condition,with intact T-junctions, is illustrated in figure 4a; the experimental condition, where theT-junctions are modified, is illustrated in figure 4b. The stimuli were generated by aMacintosh computer and presented on a 17-inch Apple monitor. The screen was splithorizontally and the images for the left and right eye were drawn one above the other;stereo fusion was achieved by using prism glasses that displaced the images so thatthey appeared to be coming from the same location (the center of the screen). Toeliminate the `ghost' image of the left eye from the right eye and vice versa, orthogonallinear polarizing sheets were superimposed on the screen and the prism glasses. Inthe experimental stimuli, the occluders were red (luminance: 22 cd mÿ2), the horizontalstripe was black (2 cd mÿ2), and the background was light-gray (65 cd mÿ2); the polar-izers introduced a reduction by a factor of four in the luminance. The color of thehorizontal stripe was set to black in order to avoid any visible texture of the screenpixels in the test patch, which might lead to conflicting stereoscopic matches. The sizeof the images was 6 deg (length of black stripe); the disparity of the occluders was24 min of arc. Viewing distance was 60 cm.

(4) In the case of figure 3b, a new T-junction is formed which is locally consistent with occlusionof the white (background) surface. In the experiments below, additional stimuli are used tocontrol for the possibility that the conflict between this local cue and global cues (eg relatability)is responsible for the effects reported here.

(a) (b)

Figure 3. The E-shaped occluders separate the black horizontal stripe into three regions; the centralregion, termed the test patch, may or may not complete perceptually with the flanking regions,depending on the structure of the T-junctions. (a) The local structure of the T-junctions is con-sistent with occlusion of the E-shaped surfaces over the black stripe. (b) The shape of theoccluders was changed slightly so that the local structure of the T-junctions no longer indicatesa generic occlusion situation of the E-shaped surfaces.

The role of junctions in surface completion and contour matching 343

2.1.2 Procedure. There were two groups of observers. In group A, observers were askedto view each stimulus until they indicated that they reached satisfactory stereo fusion.This usually took a few seconds; the total viewing time was restricted to 2 min but noneof the subjects reached that limit. They were then asked to ` describe all of the parts of thepicture'' in their own words, and their responses were noted. If they did not volunteerexplicit information about the perceived depth of the test patch, they were asked: ` lookat the black patch in the middle of the picture and tell me where you see it in depth;is it in the same depth plane as any of the other parts of the picture?''. Their responseswere again noted. Finally, the observers were asked if they experienced bistability. Theexperimental and control stimuli were presented in counterbalanced order amongobservers. Half of the observers were run by the author and the other half (selectedrandomly) were run by a research assistant who was na|« ve about the hypothesis beingtested. The responses of all observers were written down verbatim, and finally theresearch assistant scored all the subjects for whether they judged the test patch to bein front or in the back. In group B, observers were shown the same stimuli, but thistime (i) they were asked to fixate at the center of the test patch and at the top blackdot (above the test patch), alternately in counterbalanced order, and (ii) they wereonly asked to make one forced-choice judgment: ` is the central black patch in front orin the back?''. All of the subjects in group B were run by the author.

2.1.3 Observers. Twenty observers participated in group A, and ten in group B; theywere undergraduate students who were na|« ve about the purpose of the experiment. Allhad normal or corrected-to-normal vision and normal stereo vision. They were eitherpaid or given course credit for their participation.

(a)

(b)

Figure 4. Illustrations of the stimuli used in experiment 1. For this and all subsequent stereo-grams, to obtain appropriate stereoscopic fusion diverge the two rightmost images or converge(cross-fuse) the two leftmost images. (a) The control stimulus (intact T-junctions); observersreport perceiving the test patch in the back, completing with the flanking regions. (b) The exper-imental stimulus (manipulated T-junctions); observers report perceiving the test patch in front,flush with the occluders.

344 N Rubin

2.1.4 Results and discussion. For the experimental stimulus (illustrated in figure 4b),all twenty observers of group A and all ten observers in group B judged the test patchto be in front. The fixation condition did not affect results of group B. All the observ-ers in group A reported their percept was stable, ie they did not experience bistability.

For the control stimulus (illustrated in figure 4a), nineteen out of the twenty observersin group A judged the test patch to be in the back, at the same depth plane as thereference points and the two other black segments. Those observers reported that theirpercept was stable. Only one subject reported that the depth of the test patch wasbistable, and chose `front' as predominant. In addition, most of the subjects in thatgroup (thirteen) volunteered statements that indicated they perceived the test patch tocomplete perceptually with the two flanking black regions (eg ` there is a long horizontalstripe behind two red objects''; subjects were not directly asked about perceptual com-pletion, only about depth). In group B, all ten observers judged the test patch to be in theback when fixating the top black dot. When fixating the patch itself, nine judged it tobe in the back and one subject chose front. The manipulation of the structure of theT-junctions therefore had a decisive effect on the perceived depth of the test patch.

The two different percepts reported by subjects for the experimental and control stimuli(figures 4b and 4a) are possible because the stimuli are ambiguous with respect to thestereoscopic depth of the test patch. The ambiguity rests in the fact that the vertical edgesof the test patch may or may not be stereoscopically matched between the images of thetwo eyes.(5) To directly see the result of matching the vertical edges of the test patch betweenthe two images, examine figure 5a. In this stereogram, all of the items were removedexcept for the test patches, reference points, and background frame. The test patch isclearly seen in the front depth plane. Indeed, inspection of the left-eye and right-eyeimages reveals that the patch has crossed disparity with respect to the frame and refer-ence points. Note, however, that the location of the patch relative to the frame in each

(5) A similar stereogram to that shown in figure 4a is presented by David Marr (1982, p. 287) andattributed to Andrew Witkin. Marr mentions the ambiguity in the stereoscopic depth of the testpatch but does not comment on the role of junctions.

(a)

(b) (c)

crossed disparity(`front')

zero disparity

extrinsic edges ^undetermined disparity

Figure 5. (a) When the occluders and flanking patches are removed from the stereograms infigure 4, the test patch appears stereoscopically in front. (b) and (c) The fact that the testpatch was judged to be in the back when the T-junctions were intact (figure 4b, experiment 1)indicates that its vertical edges were not used for stereoscopic matching, because they wereclassified as extrinsic.

The role of junctions in surface completion and contour matching 345

monocular image is identical in all three stimuliöfigures 4a, 4b, and 5a. The fact thatthe test patch did not appear in the front plane in figure 4a suggests, therefore, that itsvertical edges were not used for stereoscopic matching in that case. This is readily under-stood by considering the occlusion interpretation of the stimulus. In this interpretation,the scene is considered as made of two `layers', one layer containing a horizontal blackstripe in the back, and another layer containing the occluders in the front. In this case,the only edges which are relevant for determining the depth of the black stripe are theouter edges of the two flanking black regions (see figure 5b). The vertical edges of thetest patch that border the occluders do not mark true surface boundariesöthey are anartifact of occlusion. To use the terminology introduced by Shimojo et al (1989), theseedges are extrinsic. Therefore, they should not be matched stereoscopically between thetwo images (figure 5c). The depth of the test patch therefore cannot be determineddirectly from the stereoscopic disparity of any part of its bounding contour. Note thatthe horizontal parts of the boundary carry no stereo information, as displacements ofa horizontal edge lead to no binocular disparity; therefore only the vertical edges in thestimuli considered here can be used for stereoscopic matching (see also below, figure 11).In the absence of direct stereoscopic information from bounding contours, the depth ofthe test patch defaults to the plane of the surface with which it completes (when suchcompletion is further supported by other cues; see below).

Conversely, the fact that the test patch appeared in front for the stimulus illustratedin figure 4b indicates that the vertical parts of its bounding contours were used for stereo-scopic matching. In other words, when the T-junctions were manipulated so that theyare no longer locally consistent with occlusion of the gray surface over the black one,the edges of the test patch were taken to be intrinsic (Shimojo et al 1989), and con-sequently used for stereoscopic matching. Note that the results we obtained would notbe expected if the other occlusion cuesörelatability and surface similarityöled to a com-pletion of the three disjoint black regions into a (perceptually) unitary horizontal stripe.

The above analysis suggests that the importance of T-junctions for the perceptionof occlusion and amodal completion lies in their role in resolving the border ownershipproblem: given a boundary between two regions, the generic situation is that only oneof the surfaces terminates at the border, whereas the other surface continues behind(E Rubin 1921; Nakayama et al 1995). The appropriate assignment of border owner-ship along edges is a fundamental problem in segmentation: at every luminance, color,or texture edge, the system has to decide: does it arise from an inherent change insurface property, or does it arise from occlusion of one surface over another? and ifthe case is the latter, which is the occluding surfaceöie which side `owns' the border.This ambiguity cannot be resolved locally at smooth parts of an edge,(6) but a 2-Dsingularity such as a T-junction offers a likely solution. The resolution of the border-ownership ambiguity is at the heart of the mechanism underlying amodal completion.It will play a central role in the theoretical framework presented in section 3.

2.2 Experiment 1a: can a Y-junction serve as a local cue for occlusion?One potential criticism of experiment 1 is that by changing the shape of the occludersto eliminate the original T-junction, it was turned into another T-junction, one whichlocally signals occlusion of the background surface. Perhaps this inconsistency betweenlocal and global occlusion cues led to the results observed. The stimuli presented infigures 6a and 6b address this point. The shape of the vertical occluders was changedso that the test patch was a hexagon, again identical in shape and position in the twostimuli. But, whereas in figure 6a the intersection between the bounding contour of theoccluders and the stripe still produces a T-junction (recall that the intersecting bounda-ries do not have to meet at a 908 angle for the junction to be labeled a T-junction),

(6) Unless there is independent depth information, such as stereo or relative motion.

346 N Rubin

in figure 6b the intersection has a different local structure and produces instead aY-junction. This junction shares an important feature with the T-junction geometry,in that it is locally consistent with the gray surface being in front. However, it is notrepresentative of the generic occlusion case, because, in order for the cusp in thebounding contour of the occluding bar to fall right on the edge of the horizontalstripe, the observer has to be at a very particular vantage point. Therefore it is likelythat the visual system did not evolve to regard Y-junctions as a special case ofT-junctions, and therefore that the pair of stimuli in figures 6a, 6b would produceanalogous results to those observed for figures 4a, 4b.

2.2.1 Methods. Methods were similar to those described in experiment 1 (group A) exceptthat only ten subjects participated in this experiment.

2.2.2 Results and discussion. As predicted, observers report seeing the patch in theback for the stimulus illustrated in figure 6a and in front for figure 6b. This was thereport of all ten subjects.

These results suggest that the emergence of perceptual completion is preventedeven when the local junctions structure is consistent with the gray surface being infront, as long as the junction created (a Y-junction, see stimuli in figure 6b) is differentfrom what would be created in the generic case of occlusionöie a T-junction. Indeed,the stimuli illustrated in figure 4b and in figure 6b are both consistent with an inter-pretation of occlusion of the gray bars in front of a unitary horizontal black stripe.In both cases, however, the image represents an accidental caseöone that would becreated only from a unique, singular point of view. This means that theories which statethat the perceptual interpretation of an image will be the one most consistent witha generic viewpoint (Koenderink and van Doorn 1976; Biederman 1985; Malik 1987;

(a)

(b)

Figure 6. The stimuli used in experiment 1a. (a) Intact T-junctions are formed at the inter-sections between the occluders and the hexagonal test patch; observers report that the testpatch is in the back, completing with the flanking regions. (b) The shape of the occluders waschanged slightly so that the T-junctions turned into Y-junctions. While the Y-junctions areconsistent with an occlusion of the gray surface both locally and globally, they do not representthe generic occlusion situation. Experimentally, observers perceived the test patch in this caseto be in front, indicating that Y-junctions do not launch completion processes.

The role of junctions in surface completion and contour matching 347

Nakayama and Shimojo 1992; Freeman 1994; Albert and Hoffman 1995) will thereforealso predict the results observed in experiments 1 and 1a. In fact, some authors haveused generic-viewpoint arguments to motivate the importance of junctions for percep-tual completion (Shipley and Kellman 1990). Therefore, the generic-viewpoint principleand a statement that junctions are important for perceptual completion may be equiv-alent at the computational level. However, an explanation based on junction structureis different, because it readily lends itself to implementation in neural-like modelswhich encode the image on a set of local, small receptive-field units. Indeed, as will beshown later, an implementational-level analysis will be more adequate to explain theresults in the case of L-junction and illusory contours.

It has already been noted that, although experiments 1 and 1a involved stereoscopicimages, the effect of the T-junctions on the depth-ordering of the surfaces around itdoes not depend on there being stereoscopic depth in the image. Indeed, the readermay have observed that the effect of manipulating the T-junctions can be seen alreadyin the monocular images, ie those shown in figure 3. Informally, many observers reportthat in figure 3a they perceive the test patch to complete behind the vertical gray barsand link with the flanking black regions, whereas in figure 3b the test patch appearsto be in front, or at the same depth plane as the gray bars. However, the difference inperceived depth is obviously not as compelling as in the stereoscopic display, andsubjects who are not experienced psychophysical observers often find it hard to giveanswers about the perceived depth of surfaces in non-stereo displays. Therefore, stereoand motion were used in order to enhance the effect and facilitate responses fromna|« ve observers. The stimuli used allow one to ask questions that subjects can answerunambiguously: Is the test patch in front or in the back? Was it moving or not? Butthe effect of T-junctions on surface completion and border ownership is valid alreadyfor a monocular image, as is indeed observed.

2.3 Experiment 2: T-junctions and edge matching in motion perceptionFinding the correct corresponding features between images is a central problem notonly for stereo (between images of the two eyes), but also motion (between successiveimages in time). Experiment 2 was designed to examine the role of T-junctions incompletion and edge matching in motion perception. The stimulus configuration isillustrated in figure 7. Similar to the previous experiment, there are two E-shapedoccluders in front of a horizontal black stripe. The occluders move from right to left(or vice versa) along a horizontal trajectory, short enough so that the two flankingblack patches are always visible. The horizontal stripe is static throughout the motionsequence. In analogy to the stereo situation (experiment 1), this stimulus is ambiguousabout the motion of the central (`test') black patch. One possibility is that the patchis moving horizontally together with the two vertical bars. This is the most `literal'interpretation of the scene, where every edge is matched between successive framesöand, in particular, the edges of the test patch are matched, as if indicating motion ofreal surface boundaries. In contrast, according to the occlusion interpretation, the testpatch is part of a unitary stripe in the back and its boundaries are therefore extrinsic.As the occluders move, parts of the black stripe are covered and other parts areexposed (`accretion and deletion', cf Gibson 1968), but there is no actual motion ofblack `stuff '. Whether or not the test patch will be perceived as accreting and deleting(as opposed to moving) therefore depends on whether or not it is perceived to completebehind the occluders or, in other words, whether its edges are perceived as extrinsicor intrinsic. Therefore the perception of motion of the test patch was studied in fourcases: with intact local T-junction cues, with occluders shaped like those in figures 4aand 6a, and with manipulated T-junctions, by changing the occluders' shapes intothose illustrated in figures 4b and 6b.

348 N Rubin

2.3.1 Stimuli. The stimuli were generated by a Silicon Graphics Indigo II computerand presented on a 20-inch Mitsubishi monitor. Refresh rate was 72 Hz. The occluderswere red (luminance: 12 cd mÿ2), the horizontal stripe was black (0 cd mÿ2), and thebackground was gray (28 cd mÿ2). The size of the display was 13.2 deg (width of blackstripe) by 9 deg (height); viewing distance was 100 cm. The vertical occluders traverseda distance of 9 deg at a speed of 6.4 deg sÿ1.

2.3.2 Procedure. On each trial, the stimulus appeared static for 250 ms and then theoccluders were set into motion, translating horizontally from right to left or vice versa,in a random order. The motion sequence lasted 1400 ms; eye movements were notrestricted. At the end of the trial, the observer was prompted to respond whether s/heperceived the test patch to move or be in the back, covered and uncovered by the verticaloccluders. The four types of stimuli were presented 12 times each, in a pseudo-randomized order.

2.3.3 Observers. The author and two other observers, who were na|« ve about the questionbeing studied, participated in the experiment. All had normal or corrected-to-normalvision. The na|« ve observers were paid for their participation.

2.3.4 Results. Figure 8 shows the mean probability of responding ` test patch moved''as a function of stimulus type, organized in terms of whether or not a T-junction waspresent. The results indicate that, similarly to the case of stereo, an intact T-junctionstructure is necessary for amodal completion to occur, so that the test patch is seenas part of the stripe in the back rather than as a distinct, moving surface. When thelocal junction structure was changed, the test patch was seen as moving 92% of thetime for the stimuli shown in figures 4b and 6b, compared to only 10% for the stimuliin figures 4a and 6a. This happened although the relatability and surface similaritycues were the same for the two cases of junction structure, and again suggests thatthese cues alone are not sufficient to lead to an amodal-completion interpretation.

time t

time t� Dt

Figure 7. Illustration of the stimulus used in experi-ment 2. The occluders moved along a horizontaltrajectory; their path was short enough so that theynever crossed the outer boundaries of the occludedstripes. The motion of the test patch (the regionenclosed between the two occluders) is ambiguousin this stimulus. It could be interpreted as movingtogether with the occluders, or it could be seen asbeing continuously accreted (here, on the right) anddeleted (on the left), owing to occlusion. In this latterinterpretation, no perceived motion of the test patchis expected. Observers report seeing no motion ofthe test patch for the case of intact T-junctions,ie when the occluders are shaped like here or likethose in figure 6a. In contrast, when the T-junctionswere manipulated (occluders' shapes like those infigures 4b and 6b), the test patch was judged asmoving most of the time.

The role of junctions in surface completion and contour matching 349

3 Local and global occlusion cues and the emergence of amodal completion: a theoreticalframeworkIn the experiments reported above, it has been found that T-junctions can have adecisive effect on the perceived stereoscopic depth and motion of a test patch whichwas ambiguous with regard to those attributes. The explanation offered for the stereo andmotion experiments followed similar lines, namely that the T-junctions were importantfor the perceptual emergence of amodal completion, via affecting the resolution ofthe border-ownership ambiguity of the edges of the test patch. Furthermore, althoughthe other occlusion cuesörelatability and surface similarityöremained unchanged,these cues by themselves did not lead to a perceptual completion of the test patch withthe `relatable' surfaces. There are several studies, however, that show that changes inrelatability can have a significant effect on the perception of amodal completion (Kell-man and Shipley 1991; Wouterlood and Boselie 1992; Yin et al 1997; but see Andersonand Julesz 1995 for a discussion of completion in the absence of relatability; see alsobelow, figure 11). How can we reconcile the present results with the results of thosestudies? An explanation can be offered that rests on the distinction made earlier betweentwo types of occlusion cuesölocal ones (such as T-junctions) versus global ones (suchas relatability). Local and global occlusion cues affect the perception of amodal com-pletion at different stages of visual processing. The advantage of such a computationalstrategy becomes most evident when issues of implementation in a physical networkare considered. Here is an outline how a network comprised of small receptive-fieldunits with only local interactions (eg nearest-neighbor) can use local occlusion cues tofacilitate the process of searching for global occlusion cues in the image, ultimatelyusing occlusion cues at all levels (local and global) to arrive at the final segmentationinterpretation.. The detection of a T-junction generates a local pattern of activation, encoding a biasthat the termination of the T-stem edge is extrinsic, (ie that it arises from occlusionby the surface on the T-head side, denoted i in figure 9a) and that the surfaces onthe T-stem side (ii and iii in figure 9a) continue behind the occluder (see figure 9b).

. This local pattern of activation launches a process of propagation of signals acrossthe network. The signals originate at the location of the junction, propagate out-wards in the direction of the T-head side, and encode a bias that the edge andsurfaces on the T-stem side continue behind the occluder (see figure 9c).

. The signals encoding bias for the occlusion interpretation will be enhanced if theyare met by consistent signals originating from other local cues in the image. Thisis the stage where global occlusion cues such as relatability and surface similarityplay a role (see figure 9d). Conversely, if the signals generated by the local junction

100

80

60

40

20

0

`Patch

mov

ed''respon

ses=%

Intact NoT-junction T-junction

E-shapedoccludersHexagonalpatch

Figure 8. Results of experiment 2. Thepercentage of times that subjects perceivedthe test patch to move is plotted against thetype of stimulus (36 trials per datum point).Diamond symbols refer to stimuli likethose illustrated in figures 4a (`intactT-junctions') and 4b (`no T-junctions');square symbols refer to stimuli like thoseillustrated in figure 6. Standard errorsacross the mean responses of the threeobservers are indicated.

350 N Rubin

are not mutually enhanced by consistent signals from other parts of the image,they will tend to decay, leading to a rejection of the occlusion interpretation.Whether or not the final global percept will be consistent with the local occlusioninterpretation will therefore depend on information from other parts of the image.

According to this scheme, T-junctions play a crucial role because they launch the processthat results in amodal completion of a surface behind an occluder. Of course, the ulti-mate percept will be a result of a complex process which involves many other factors(eg other occlusion cues), and possibly more than one iteration, until a self-consistent,ecologically valid interpretation of the scene is reached. Nevertheless, the fact that aT-junction is a local cue for occlusion makes it reasonable to give it a privileged rolein the process: in a distributed system that represents the visual information on a setof units with small receptive fields, the ability to detect occlusion cues locally can bea great advantage. In particular, it allows the analysis of relatable edges and surfacesto be concentrated on those which terminate with a T-junction. This strategy offers con-siderable computational benefits, because it restricts the space of possible linkings whichmay be rather large a priori. To illustrate this, refer back to figure 4a. In addition tothe relatable edges of the occluded black stripe, there are several other relatable (in thissimple case, collinear) edges in the scene: those bounding the two occluding (E-shaped)surfaces. Without knowledge about local occlusion cues, every pair of relatable edgeswould need to be processed as a potential for contour/surface linking. But if linkingprocesses are restricted to those launched by local occlusion cues, as suggested above,the search space for relatable edges is dramatically reduced: in the case of figure 4a,it leaves only the contour fragments bounding the horizontal stripe, and this is repre-sentative of the generic case, where relatability of occluded edges will be accompaniedby the existence of T-junctions. Conversely, in figure 4b the relatability of the horizon-tal stripe fragments is not processed because of the absence of T-junctions to launchthe process, leading to a lack of perceptual completion, as is observed experimentally.

(a) (b)

(c) (d)

iii

iii

Figure 9. The launch of global completion processes by local T-junctions. (a) and (b) The junctionstructure (a) suggests that surface i is in front of surfaces ii and iii. (c) The local occlusionhypothesis generated by the detection of the junction launches a process of propagating signalsto complete the T-stem edge and surfaces ii, iii behind surface i. (d) The completion signals maybe enhanced if they are met by signals from relatable edges and surfaces.

The role of junctions in surface completion and contour matching 351

An important property of the proposed theory is that it allows for the emergenceof perceptual completion using only units with small receptive fields and interactionsbetween them (for similar approaches see Saund 1999 and for the case of amodalcompletion see Kumaran et al 1996; Williams and Jacobs 1997; see also below). This isin contrast with theories which rely on the existence of large receptive-field units,which are activated in response to relatable inducers (with or without demanding theco-existence of local junction cues; see, eg, Grossberg and Mingolla 1985; Heitger andvon der Heydt 1993). A concrete implementation of the proposed scheme in a modelwhich simulates human visual segmentation requires decisions about the nature andrange of local signals that give rise to the propagation process. For example, a T-stemedge, ie one which is locally assumed to terminate only owing to occlusion, may con-tinue in infinite ways. A reasonable approach is to generate a completion field' whichfavors straight, smooth continuation, and falls off in strength with the curvature ofthe continuing contour, as shown schematically in figure 9c, but the exact form of thecompletion field may differ between models (cf Ullman 1976; Field et al 1993; Williamsand Jacobs 1997; Finkel and Yen 1998). Similarly, the propagation of signals about thecontinuation of surfaces (as opposed to edges) behind occluders may be implementedin various ways, and in particular depends on how surface properties are represented inthe model. In a model which attempts to be neurally plausible, there is also an issuehow to represent signals about amodal completion. In contrast with the case of modalcompletion, where researchers reported the existence of cells that fire in response toillusory contours (von der Heydt et al 1989; Peterhans and von der Heydt 1989; Bakinet al 1998), in the case of amodal completion the mediation of signals may need totake another form, eg via `priming' connections which do not elicit explicit firing untilcertain conditions are met (Ullman 1995).

The results of experiments 1 and 2 have implications also for models that areconcerned with the issue of matching features for stereo or motion. The different perceptobserved for the stimuli in figures 4a versus 4b suggests that stereoscopic matchingdoes not operate indiscriminately on all the edges in the image: edges are not matchedif they are classified as extrinsic. Similarly, for the case of motion (matching across time),moving edges are matches across images only if they are classified as intrinsic. Note,however, that this does not necessarily imply that the resolution of the border-ownershipambiguity takes places before stereo or motion matching. Rather, it is more likely thatthey take place concurrently, as part of a (possibly iterative) process aimed at arriving atthe most stable interpretation of the scene. A plausible way to implement an interactionbetween stereo or motion matching and resolution of border-ownership ambiguities isto build into the system a constraint that an edge can only be used for matching asthe bounding contour of one surface, from one of its sides, but not from both. Whichside will be used for matching will depend on various image cues, such as the localjunction structure (as in figures 4 and 6), stereoscopic disparity of the edges, etc. Ifthe scene contains abutting surfaces where there are no image cues about the borderownership of the edge which separates them, these surfaces may turn perceptuallyinto a single surface. Such is the case of the border between the E-shaped occludersand the test patch in figure 4b, as well as in the case of random-dot stereograms(Julesz 1964, 1971) and kinematograms (Braddick 1974), where small squares that areused as matching cues cohere into a single, perceptually `smooth' surface.

Above, it was proposed that global cues such as relatability come into the pictureafter linking processes were launched by the detection of T-junctions. What happens ifthe T-junctions are locally consistent with occlusion, but there are no global cues forocclusion? Figure 10 shows that, in the absence of any other occlusion cues, T-junctionsmay not be sufficient to support perceptual completion which will `push' a surface toa different depth plane. In the stereogram in figure 10a, a gap was introduced between

352 N Rubin

the flanking patches and the E-shaped occluder, while everything else in the imagewas left as in figure 4a. In figure 10b, the flankers were moved upward slightly so as toperturb relatability. In both cases, observers report a difficulty to determine whetherthe test patch is at the back or at the same depth plane as the E-shaped surfaces,(7)

as the percept seems to alternate between the two. However, this should not be takento imply that the classification of T-stem surfaces as extrinsic is suppressed in theabsence of relatable edges. Indeed, using an apparent-motion paradigm, Shimojo andNakayama (1990) showed that a small tab-shaped surface whose edges are the stems of aT-junction continues perceptually behind the T-head surface by as much as two thirdsits length. The indeterminate perceived stereoscopic depth of the test patch in figure 10likely results from the absence of a relatable surface with a definite stereoscopic depth(ie with contours that can be matched between the two images).

The situation in figure 10 is unusual, because the existence of T-junctions in themonocular images is not supported by a generic stereoscopic T-junction structure.As mentioned earlier, horizontal edges provide no stereoscopic information. Once thevertical edges of the test patch have been rendered extrinsic owing to the T-junctions, itsstereoscopic depth is therefore entirely undetermined. The stimulus was constructedspecifically this way, in order to directly compare the effect of (monocular) T-junctionswith global occlusion cues. But the generic case in stereoscopic (or moving) images isquite different: when the projection of the T-stem edge on the frontoparallel plane is nothorizontal, and in particular when the edge is curved, local stereoscopic information will

(7) Because of the difficulty to elicit confident responses, this stimulus was not used in an experimentwith na|« ve observers. Six experienced psychophysical observers who were shown the stimulusreported that it did not lead to a stable percept, and that the perceived depth of the test patch wasgreatly affected by direction of gaze and attention.

(a)

(b)

Figure 10. The role of relatability in perceptual completion. When the test patch is no longerrelatable with the flanking regions [(a) and (b)] it appears stereoscopically in front, even thoughthe local T-junction structure is consistent with it being occluded. This suggests that corroborat-ing evidence for the local occlusion hypothesis from global image cues such as relatability playsa role in perceptual completion.

The role of junctions in surface completion and contour matching 353

be enough to disambiguate the depth relationships. This is shown in figure 11: there, theshape of the occluded stripe was changed so that its edges were no longer perfectlyhorizontal. In this case, the central black patch is seen in the back stereoscopic planeboth when global relatability cues are present (figure 11a) and when they are absent(figure 11b). To appreciate the contribution of stereoscopic cues to determining theperceived depth of the test patch, compare figure 11c with figure 5a. In both cases, theoccluders and the flankers were removed from the image, leaving only the test patches andthe reference points. But, whereas in figure 5a the test patch is seen in the front plane,reflecting the complete lack of stereoscopic cues to indicate that it belongs to the backplane, in figure 11c the stereoscopic matching generates a percept of the test patchpartially occluded behind two illusory vertical `walls' (cf Anderson and Julesz 1995).

In the next section (about illusory contours) we shall see that when L-junctionsare eliminated from the image by superimposing small occluding `buttons' on them, theperceptual emergence of illusory contours is blocked. It is therefore interesting tonote here that a similar image manipulation is not effective in the case of T-junctions,

(a)

(b)

(c)

Figure 11. In general, the T-junctions in the monocular images will be accompanied with stereo-scopic local occlusion cues, and those may be sufficient to push the test patch to the backplane (a). In this case, global occlusion cues such as relatability may not be necessary (b). Thestereogram in (c) demonstrates the existence of stereoscopic occlusion cues which are separatefrom the monocular T-junctions. Note, however, that the two sets of cues should be in agreement.In (b), compare the correct' stereogram (right-hand pair when diverging) with the stereogramobtained when the eyes are switched (left-hand pair when diverging), where the stereoscopicdepth relationships are reversed while the monocular T-junctions remain unchanged.

354 N Rubin

as can be seen in figure 12. Although all four T-junctions are covered by the occludingbuttons which are stereoscopically nearest to the observer, the test patch is neverthelessoften seen to complete in the back plane, together with the flanking patches. Note, how-ever, that superimposing the buttons creates new T-junctions. It is therefore likely thatthese new T-junctions launched completion processes causing the edges to continue them.

Finally, although we have emphasized the role of T-junctions (either monocular orstereoscopic) in resolving the border-ownership ambiguity, it should be noted that theyare by no means the only relevant image cue. There are many examples of imagesthat contain no T-junctions at all, and yet a clear resolution of border ownership isachieved. Figure 13 presents some well-known examples, where the role of size (Koffka1935), convexity (Kanizsa 1979), and parallelism (Rock 1983) may determine borderownership. In fact, even the very simple image of a white disk on a black background,which we tend to think of as entirely unambiguous, requires border-ownership assign-ment, since it could also represent a black circular aperture showing a white surfacebehind it. It is therefore evident that, while any small portion of a smooth edgedoes not hold the key for resolving border ownership, surface-based (or `region-based')interactions triggered by such borders may be sufficient in many cases.

4 L-junctions and the emergence of illusory contours and surfacesWhen the occluding surface has the same color and luminance as the background,the T-junctions become L-junctionsöa point where two parts of a continuous edgemeet at a cusp (figure 2a; see also Shipley and Kellman 1990). Although L-junctionsthat arise from the true shape of objects are common in the modern world, and caneven be found in the wild (eg the tips of leaves), occlusion may still be the main sourceof L-junctions in natural images (see footnote 3). Again, because they are local image

Figure 12. Elimination of the test-patch T-junctions by using occluding buttons is less effectivethan the method shown previously; observers report a bistable percept, where the test patchalternates between completing with the stripe in the back and being flush with the vertical occluders.

(a) (b) (c)

Figure 13. Resolution of the border-ownership ambiguity in the absence of local junctioninformation. (a) The effect of size. The smaller regions are perceived as figure (after Koffka1935). (b) Parallel regions appear as figure (after Rock 1983). (c) Convex regions appear as figure(after Kanizsa 1979).

The role of junctions in surface completion and contour matching 355

cues, it is plausible to assume that L-junctions play a privileged role in the perceptualemergence of illusory contours, compared to global image cues. Below, we show thatthis is indeed the case.

4.1 Experiment 3: L-junctions and stereoscopic illusory contours4.1.1 Stimuli. The method of presentation of the stereoscopic stimuli was the same asdescribed in experiment 1. The stimuli contained a Kanizsa square (1979) that hadcrossed (`front') stereoscopic disparity compared to the inducers. In the experimentalstimulus, L-junctions were eliminated from the monocular image of both eyes bysuperimposing small occluding buttons such that they completely covered the cusp(figure 14a). The occluding buttons were given a crossed disparity which was slightlylarger than that of the Kanizsa square, so that they appeared to be stereoscopically

(a)

(b)

(c)

Figure 14. Illustration of stimuli used in experiment 3. (a) Occluding buttons are placed on theL-junctions; observers report perceiving no illusory contours along the horizontal boundary ofthe Kanizsa square. (b) When the buttons are moved outwards to expose the L-junctions, observersreport perceiving clear illusory boundaries. (c) Illusory contours are reported also when theoccluding buttons are placed on the contour itself.

356 N Rubin

nearest to the observer. Occluding buttons were introduced only over the L-junctionsassociated with the horizontal boundaries of the illusory square, again because hori-zontal edges do not provide stereoscopic information, and therefore we could directlycompare the effect of (monocular) T-junctions and other occlusion cues. There weretwo control stimuli. In the first control stimulus, the occluding buttons were movedslightly outwards, so that the tip of the L-junctions was exposed (figure 14b). In thesecond control stimulus, there was one occluding button, placed at the (stereoscopic)center of the space between the L-junctions, so that the junctions were again com-pletely exposed (figure 14c). The purpose of this control stimulus was to establish thatthe emergence of perceptual illusory contours is not blocked by any interruption alongthe path of the contour, but only specific interruptions that include the L-junctions.In both control stimuli the stereoscopic depth of the occluding buttons was the sameas in the experimental stimulus. The background and illusory square were white(115 cd mÿ2), the inducing elements were black (2 cd mÿ2), and the occluding buttonswere magenta (27 cd mÿ2). The size of the stimulus was 5.2 deg (diameter of inducers1.7 deg; side of Kanizsa square 3.5 deg; length of illusory gap 1.7 deg; diameter ofbuttons 0.5 deg). The disparities of the Kanizsa square and the occluding buttons were15 and 18 min of arc, respectively. Viewing distance was 60 cm.

4.1.2 Procedure. Observers were shown an example of an illusory-contour stimulussimilar to that shown in figure 2a printed on paper, and received a brief explanationabout the phenomenon of illusory contours. All subjects reported seeing `white-on-white'borders in the `illusory' parts of the bounding contour of the illusory surface. Theywere then asked to view the experimental and control stimuli until they reached satis-factory stereo fusion. Observers were asked to ` describe all of the parts of the picture''in their own words; after they mentioned the (Kanizsa) square, they were asked to` examine each of its sides in turn and report whether you see a white-on-white borderthere''. The order of presentation of the three stimuli was pseudorandomized acrossobservers.

4.1.3 Observers. Twenty observers participated in the experiment; they were the samesubjects who participated in experiment 1 (group A).

4.1.4 Results and discussion. For the experimental stimulus (figure 14a), all twentyobservers reported that, upon inspecting each of the sides of the illusory squareseparately, they did not perceive a white-on-white border along the horizontal sides ofthe square. At the same time, they reported seeing clear white-on-white borders along thevertical sides. For the control stimuli (figures 14b and 14c), all twenty subjects reportedseeing white-on-white borders at all four sides; in the stimulus illustrated in figure 14c,

Figure 15. Here, the occluding buttons were placed so that they expose the L-junctions in onlyone monocular image at a time, and illusory contours are still observed. Monocular L-junctionsare therefore sufficient to generate an illusory contour.

The role of junctions in surface completion and contour matching 357

the horizontal illusory sides were reported to be ` interrupted by the small circle infront'' but ` visible around it''.

The results underscore the importance of L-junctions for the perceptual emergence ofillusory contours. Specifically, the difference in the reported percept between the exper-imental and control stimuli suggests that the local sharp discontinuity is necessary. Inthe absence of L-junctions, the formation of an illusory contour which is perceptuallyexperienced (`modal completion') does not take place, even when the observer reportsseeing the global surface. The difference between the reported percepts in the experimentaland control stimuli demonstrates that, for the case of modal completion, too, relatabilityis not sufficient to launch the completion process. Note that, because of the differentplacement of the occluding buttons in the two stimuli, the length of relatable horizontaledges was in fact greater in the experimental than in the control stimulus, and yet illusorycontours were perceived for the latter, but not the former.

Our results are consistent with those of Minguzzi (1987) and Shipley and Kellman(1990), who found that the perception of illusory contours was greatly attenuated whenthe L-junction discontinuities were rounded off. There is a basic difference, however,between our stimulus and those reported previously: when the L-junctions are eliminatedby rounding the corners, an occlusion interpretation is no longer plausible from anecological point of view. This is because, for an occlusion interpretation to exist in thecase of a `rounded' L-junction, the tangents of the bounding contours must be parallelat the point of intersection. This will occur, if at all, only from a specific and uniquevantage point, ie it is not a generic situation. On the other hand, for the stimulus weused (figure 14a), an interpretation of the scene as containing two layers of occlusion(a Kanizsa square and occluding `buttons' in front of it) is valid from a continuum ofpoints on the viewing sphere, ie it is consistent with the generic-viewpoint hypothesis.

Why was this interpretation nevertheless not observed in experiments 3 and 4?To answer this we need to consider how the brain may arrive at a given scene inter-pretation, mechanistically. One possibility is that the brain implements knowledgeabout junctions (or, more generally, searches for an interpretation consistent with ageneric viewpoint) via a process that analyzes the scene globally. This hypothesis, how-ever, would predict that illusory contours should have been observed(8) in experiments3 and 4öcontrary to what has been found here. In contrast, these results are entirelyconsistent with the proposal, that the implementation of the role of junctions is local,and that it is their detection that launches more global completion processes. Theremay be several reasons why this specific form of implementation is used. A naturalpossibility is that neural wiring constraints favor local computations, whenever possible.Another possible reason, mentioned earlier, is that constraining the search over thespace of permissible global interpretations to those suggested by local occlusion cuesreduces the computational complexity of the problem. Whatever the reasons are (andthere are likely to be multiple ones), the fact that local junction structure is (generically)so tightly coupled with global occlusion makes this locally based implementationstrategy a very effective one. Nevertheless, under special circumstances, such as thosecreated in the experiments described here, this strategy may fail, thereby exposing theunderlying mechanisms of the system.

In the stereogram in figure 15, the occluding buttons were placed so that one of themexposed the L-junction in one monocular image, while the other junction was stilloccluded. Horizontal illusory contours are still observed, suggesting that monocularL-junctions are sufficient to generate an illusory contour. Furthermore, as in the case

(8) Strictly speaking, a prediction based on the generic-viewpoint hypothesis can be made only afteranalyzing the entire space of possible interpretations and attaching a probability measure to each.Nevertheless, one may conjecture that the likeliest interpretation will in fact involve two layers ofocclusion, and therefore predict illusory bounding contours.

358 N Rubin

of our T-junction stimuli, it should be noted that the effects observed do not arise from thestereoscopic disparity in the image. The purpose of using a stereoscopic display was toenhance the perceived (or not) illusory contour and thereby make the perceptual judg-ments easier. But the information contained at the manipulated (occluded) L-junctionscannot be affected by the addition of stereo to the image, because horizontal edges do notcontain stereoscopic information. Indeed, some readers may have observed that a differ-ence in the appearance of illusory contours can be seen already in the monocular imagesof figures 14a versus 14b. Nevertheless, we found that inexperienced psychophysicalobservers were more confident giving responses about whether or not they perceivedillusory contours when those were enhanced by stereo or motion.

4.2 Experiment 4: L-junctions and motion-induced illusory contoursThe perception of illusory contours can be greatly enhanced by the addition of motioninto the scene. In the following experiment, we used motion to enhance an illusoryKanizsa square, and then eliminated some of the L-junctions and tested the effect onthe perceived illusory contours.

4.2.1 Stimuli. The stimuli consisted of a Kanizsa square which appeared to movesmoothly along an elliptical path (see figure 16a). The inducing elements appearedstatic, and the motion of the Kanizsa square was generated by smoothly changingthe shape of the corners cut from the inducing circles. In the experimental stimulus,L-junctions were eliminated from the image by superimposing small occluding buttonssuch that they completely covered the cusp (figure 16b). The motion of the Kanizsasquare was such that the L-junctions were covered at all times. Occluding buttons wereintroduced only over the L-junctions associated with the horizontal boundaries of theillusory square. In the control stimulus, the occluding buttons were moved slightly out-wards, so that the tip of the L-junctions was exposed (figure 16c); the L-junctionswere visible at all times. The occluding buttons were static in both the experimentaland the control stimulus. The stimuli were generated by a Silicon Graphics Indigo IIcomputer and presented on a 20-inch Mitsubishi monitor. Refresh rate was 72 Hz.

(a)

(b) (c)

Frame 0 Frame 14 Frame 28 Frame 42

Figure 16. Illustration of the stimulus used in experiment 4. (a) A Kanizsa square movedcontinuously along an elliptical trajectory. (b) and (c) Occluding buttons were superimposed sothat they hid (b) or exposed (c) the L-junctions. Observers report perceiving moving horizontalillusory contours in case (c) but not in case (b).

The role of junctions in surface completion and contour matching 359

The background and the moving Kanizsa square were white (114 cd mÿ2), the inducingelements were black (0 cd mÿ2), and the occluding buttons were magenta (27 cd mÿ2).The size of the stimulus was 10.5 deg (diameter of inducers 3.5 deg; side of Kanizsasquare 7 deg; length of illusory gap 3.5 deg; diameter of buttons 1 deg). Viewing dis-tance was 60 cm. The elliptical motion trajectory of the Kanizsa square was 1.3 deg(horizontal axis) by 0.9 deg (vertical axis). Average linear speed was 7 deg sÿ1 (1 fullrevolution in 1 s).

4.2.2 Procedure. On each trial, the stimulus appeared static for 250 ms and then theKanizsa square was set into motion, moving along an elliptical path in clockwisedirection. Observers were requested to ` examine each of the sides of the square inturn and report whether you see a white-on-white border there''. The motion sequencepersisted until the subject gave his/her response, but no longer than 20 s. The order ofpresentation of the experimental and control stimuli was counterbalanced across observers.

4.2.3 Observers. Ten observers participated in the experiment; they were the samesubjects who participated in experiment 2.

4.2.4 Results. For the experimental stimulus (figure 16b), nine observers reported thatthey did not perceive a white-on-white border along the horizontal sides of the square,while they did perceive such a border along the vertical sides; one observer reportedseeing white-on-white borders on all four sides. For the control stimulus (figure 16c),all ten subjects reported seeing white-on-white borders at all four sides.

The results corroborate those obtained with the stereoscopic display, that L-junctionsplay a crucial role in the emergence of illusory contours. Interestingly, in both experi-ments 3 and 4 subjects reported that there was a large square in the image, only that,when they were asked to scrutinize its sides, they could not see horizontal white-on-white edges (in the experimental stimuli). This suggests that the mechanisms that giverise to the perception of global, salient illusory surfaces and those that lead to theperception of palpable edges for these surfaces may be dissociable.

5 L-junctions and the emergence of illusory contours: a theoretical frameworkA mechanism for the emergence of illusory contours that relies on detecting L-junc-tions to launch the completion process is proposed below. The scheme extends theone proposed for the case of T-junctions and amodal completion. In analogy to whatwas described in the previous section, it is proposed that an L-junction generates alocal pattern of activation which launches completion processes which, if met by sig-nals generated in other locations in the image, may result in perceptual completion.Specifically, the local pattern of activation generated by an L-junction turns it into a`virtual T-junction': one of the edges meeting at the junction is hypothesized to con-tinue beyond the cusp, bounding a surface of the same color and luminance as thebackground, which is why one side of the virtual T-head is not observed in the image.The details of how such `virtual' junctions may be represented neurally will not bediscussed here, but signals similar to the `priming activations' proposed by Ullman(1995) to explain how top ^ down knowledge may affect perception may be good candi-dates. The virtual T-junction will then launch signal propagation processes to detectglobal occlusion cues, as described in the previous section about amodal completion.There is, however, a major difference between the cases of amodal and modal comple-tion: L-junctions carry an additional ambiguity, because locally there is no way ofknowing which of the two edges of the junction should be the T-stem and which theT-head: this is illustrated in figure 17. Therefore, at the site of an L-junction twocompeting virtual T-junctions must be generated, representing the two possible positionsof an illusory occluding surface. Each will generate its own set of completion signals,

360 N Rubin

as described for the case of T-junctions (see figure 9). Which of the two patterns ofactivation (if any) will survive to the ultimate global percept will depend on informa-tion propagated from other parts of the image. In summary:. An L-junction generates a local pattern of activation which turns it into a virtualT-junction. This represents the possibility that there is an occlusion situation, with theoccluder having the same luminance as the background (ie it is illusory). Initially,two virtual T-junctions, corresponding to the possible positions of the illusoryoccluding surface, may co-exist.

. Each virtual T-junction launches a process of propagation of local signals, as describedpreviously (see figure 9c). If the signals generated by one of the virtual junctions aremet by signals from relatable contours and/or surfaces, the pattern of activationrepresenting the junction will survive to the ultimate global percept and an illusoryoccluding surface will be perceived at the corresponding location. If neither of thevirtual junctions is supported by occlusion cues from other parts of the image, thesignals representing them will decay and the L-junction will be classified as intrinsic.This scheme is similar to that suggested by Geiger et al (1998; see also Kumaran

et al 1996), who termed the virtual T-junctions `local hypotheses'.(9) In their implemen-tation, each collection of local hypotheses is evaluated separately and given a cost'which measures the goodness of the outcome of the completion processes. Note that,given an image with p L-junctions, there are 2 p possible combinations of the localocclusion hypotheses: eg, for the case of a Kanizsa square, there are at least 256possible configurations, all of which are equally likely on the basis of local informationalone (ie global completion processes are required to decide between them). Given the

Figure 17. The detection of an L-junction generates a local-occlusion hypothesis that thediscontinuity represents a T-junction where the color and luminance of the occluding surface areidentical to those of the background (ie that there is an illusory occluding surface). Because of thelocal symmetry around the L-junction, two such local occlusion hypotheses must be generated,representing the two possible positions of an illusory occluding surface. Which of the twohypotheses (if any) will survive to the ultimate global percept will depend on information propa-gated from other parts of the image (similar to the description for T-junctions, figure 9).

(9) Although Geiger et al (1998) used the term `local hypothesis' in their model, their approach isentirely different from the cognitive' and `problem solving' approaches proposed by Gregory (1972)and Rock (1983). The concept of the local occlusion hypothesis is of a mechanistic, visually specificprocess and should not be taken in any way to indicate a cognitive process which is under theinfluence of conscious thought. Indeed, the results obtained here argue strongly against a level ofexplanation that involves thought-like processes, since the percept observed in experiment 3 was infact less `logical' than the alternative, of assuming an occlusion/completion interpretation of theimage. The fact that an occlusion interpretation was not observed speaks strongly to the point thatthe perceptual phenomena of contour and surface completion are a result of special-purpose visualprocesses that are not analogous to problem solving.

The role of junctions in surface completion and contour matching 361

rapidity with which human observers perceive illusory surfaces (Gellatly 1980; Reynolds1981; Takahashi 1991; Kojo et al 1993; Ringach and Shapley 1996), it is unlikely thatthese different configurations are evaluated serially. However, a model capable of relax-ing to the optimal completion configuration in an efficient manner remains to beformulated. One interesting possibility is that region-based mechanisms, which detectsalient regions in the images but are insensitive to the detailed boundary structure ofthese regions (or to detailed junction structure), guide the contour-completion processand thereby make it more efficient (Mumford et al 1987; Shi and Malik 1997; Sharonet al 2000). Recent evidence that salient surfaces can be detected in parallel acrossthe visual field (Davis and Driver 1994), while discrimination of their detailed boundingshape cannot (Gurnsey et al 1996; Shomstein and Rubin 1998; N Rubin et al 1999)supports this idea.

The conjecture of a role for global relatable contours in supporting virtual T-junc-tions was for the case when the border ownership of both edges of the L-junctionwas locally ambiguous. This is the case for monocular images, or in special caseswhen the illusory occluding contour is horizontal, eg in our stimulus (figure 14). But,as for the case of T-junctions, there may be local stereoscopic cues that resolve theborder-ownership ambiguity and turn L-junctions into perceptual T-junctions evenin the absence of other, global occlusion cues. The stereogram in figure 11c representssuch an example. See also Anderson and Julesz (1995) for a discussion of how stereo-scopic L-junctions may give rise to illusory contours.

6 DiscussionThe results of four experiments showed that T-junctions and L-junctions play a crucialrole in the visual processes that give rise to amodal and modal completion, respec-tively. When the junctions were removed from the image, or when their structure wasaltered, amodal completion and illusory contours were not observed, even when other,global cues for occlusion (relatability, surface similarity) were present. The results sup-port the hypothesis that junctions have a privileged role in scene segmentation, becauseof their local nature. It was proposed that the visual system uses local informationfrom junctions to constrain the multitude of global contour-completion possibilities.

A theoretical framework to account for the results was proposed in sections 3 and 5.Junctions give rise to local activation patterns that attempt to continue edges andsurfaces behind hypothesized occluders and connect them to other parts which arespatially separate in the retinal image. This scheme is obviously an oversimplification,in that it focuses on a narrow set of image cues and ignores many other factors thataffect figural organization. The theoretical framework offered here also did not providean implementation in the form of a concrete model. The purpose of this paper, how-ever, was not to provide such a concrete model but rather to establish the distinctionbetween local and global occlusion cues and put forward the proposal that this distinc-tion plays a major role in how the visual system `bootstraps' the problem of finding aglobally consistent interpretation of a scene, when at the first stages of processing theinformation is represented by a set of local, small receptive-field units. The experimentsreported here provide support for the idea that junctions, being local cues, launchglobal completion processes. Furthermore, the results are consistent with a schemethat assumes that global contour completion is achieved by propagation ( cascade') ofsignals via short-range lateral connections between small receptive-field units, ratherthan by the activation of units that are sensitive to the completed contours (ie largereceptive-field units). This proposal has significant computational advantages as it doesnot posit the existence of a multitude of specialized units dedicated to the signaling ofillusory and/or amodally completed contours. Instead, it was proposed that the repre-sentation of completed contours takes place via the activation and, more importantly,

362 N Rubin

interactions between small receptive-field units, of the same kind that represent `real'(luminance-defined) edges in the image.

A set of questions that remains open has to do with how junctions are detectedby the visual system, and what constitutes a `valid' junction in an image. In the contextof the first question, how junctions are detected, it is interesting to note the lack ofneurophysiological reports of neurons that are selective to isolated (context-free) junction-like stimuli. This suggests that the detection of junctions itself may not be performedby localized, filter-like units with receptive fields tailored to respond to junction-likelocal structure. Rather, the signaling of junctions itself may be part of interaction andpropagation processes that constantly attempt to delineate the closed boundaries ofsurfaces in the image. As a termination of an edge is detected (eg by the activationof one unit and the absence of activation of a neighboring, collinear unit), specializedconnections may seek further image cues for occlusion in the form of a neighboringcontour that may form the T-head edge.

Regarding the second question, of what constitutes a valid junction in the image,note that, although all of the stimuli and examples described in this paper involvedimages where surface edges were crisp and well-defined, this is not necessarily the casein real-world images. Luminance gradients may be blurred owing to noise (either internalor external), and therefore the distinction between situations as those illustrated infigures 4a and 4b may not be as clear-cut. It is interesting to note in this context thatthe visual system indeed appears to treat junctions at the same scale of blur as theother edges in their vicinity in the image. This is illustrated in figure 18, which is ablurred reproduction of figure 4a and of a Kanizsa square. Amodal completion andillusory contours can be seen to occur in these two images, respectively, although therelevant junctions are blurred.

(a)

(b)

Figure 18. T-junctions and L-junctions may operate at different scales, or levels of blur. The blurredjunctions in these versions of the control stimulus of experiment 1 (a) and a stereoscopic Kanizsasquare (b), which normally (ie in sharp images) would not be effective as occlusion cues, neverthelessgive rise to amodal completion and illusory contours, respectively, indicating that the scale of theluminance gradients in the scene may determine the level of sharpness expected from junctions.

The role of junctions in surface completion and contour matching 363

To summarize, a set of experimental results was presented that supported thehypothesis that junctions play a crucial role in the emergence of modal and amodalcompletion. A distinction was made between junctions, which are local occlusioncues, and global occlusion cues such as relatability and surface similarity. Being localocclusion cues, junctions are crucial for the launching of completion processes, whichsubsequently detect global contour and surface completion in the image. This schemereconciles previous results about the role of different occlusion cues, and presents aframework for future research on how to achieve global perceptual organization bya system comprised of local, interacting elements.

Acknowledgements. Thanks to Marc Albert, Davi Geiger, Barbara Gillam, Ken Nakayama,Dario Ringach, and Robert Shapley for helpful discussions, and to Jean-Michel Hupe, MichelleImber, Jonathan Pillow, Mary Pugh, and three anonymous reviewers for helpful comments ona previous version of the manuscript. Supported by the McDonnell-Pew Program in CognitiveNeuroscience and the Sloan Foundation.

ReferencesAdelson E H, Anandan P, 1990 ``Ordinal characteristics of transparency'', in AAAI Workshop on

Qualitative Vision (Cambridge, MA: MIT Press) pp 77 ^ 80Albert M K, Hoffman D D, 1995 ` Genericity in spatial vision'', in Geometric Representations of

Perceptual Phenomena: Papers in Honor of Tarow Indow on His 70th Birthday Eds R D Luce,M D'Zmura (Mahwah, NJ: Lawrence Erlbaum Associates) pp 95 ^ 112

Anderson B L, 1997 `A theory of illusory lightness and transparency in monocular and binocularimages: the role of contour junctions'' Perception 26 419 ^ 453

Anderson B L, Julesz B, 1995 `A theoretical analysis of illusory contour formation in stereopsis''Psychological Review 102 705 ^ 743

Bakin J S, Nakayama K, Gilbert C D, 2000 ` Visual responses in monkey areas V1 and V2 tothree-dimensional surface configurations'' Journal of Neuroscience 20 8188 ^ 8198

Benary W, 1924 ` Beobachtungen zu einem Experiment u« ber Helligkeitskontrast'' PsychologischeForschung 5 131 ^ 142

Biederman I, 1985 ` Human image understanding: recent research and a theory'' Computer Vision,Graphics, and Image Processing 32 29 ^ 73

Braddick O J, 1974 `A short range process in apparent motion'' Vision Research 14 519 ^ 527Chapanis A, McCleary R A, 1953 ` Interposition as a cue for the perception of relative distance''

Journal of General Psychology 48 113 ^ 132Coren S, Porac C, Theodor L H, 1987 ``Set and subjective contour'', in The Perception of Illusory

Contours Eds S Petry, G Meyer (New York: Springer) pp 237 ^ 245Davis G, Driver J, 1994 ` Parallel detection of Kanizsa subjective figures in the human visual system''

Nature 371 791 ^ 793Dinnerstein D, Wertheimer M, 1957 ` Some determinants of phenomenal overlapping'' American

Journal of Psychiatry 70 21 ^ 37Field D J, Hayes A, Hess R F, 1993 ` Contour integration by the human visual system: evidence

for a local association field'' Vision Research 33 173 ^ 193Freeman W T, 1994 ` The generic viewpoint assumption in a framework for visual perception''

Nature 368 542 ^ 545Geiger D, Pao K, Rubin N, 1998 ` Salient and multiple illusory surfaces'', in Proceedings of the IEEE

Conference on Computer Vision and Pattern Recognition, Santa Barbara (Washington, DC:IEEE Computer Society Press) pp 118 ^ 124

Gellatly A R H, 1980 ` Perception of an illusory triangle with masked inducing figure'' Perception9 599 ^ 602

Gibson J J, 1968 ` What gives rise to the perception of motion?'' Psychological Review 75 335 ^ 346Gilbert C D, Das A, Ito M, Kapadia M, Westheimer G, 1996 ``Spatial integration and cortical

dynamics'' Proceedings of the National Academy of Sciences of the USA 93 615 ^ 622Gilbert C D, Wiesel T N, 1983 ` Clustered intrinsic connections in cat visual cortex'' Journal of

Neuroscience 3 1116 ^ 1133Gregory R L, 1972 ` Cognitive contours'' Nature 238 51 ^ 52Grosof D H, Shapley R M, Hawken M J, 1993 ` Macaque V1 neurons can signal `illusory' contours''

Nature 365 550 ^ 552Grossberg S, Mingolla E, 1985 ` Neural dynamics of form perception: boundary completion,

illusory figures, and neon color spreading'' Psychological Review 92 173 ^ 211

364 N Rubin

Gurnsey R, Poirier F, Gascon E, 1996 ` There is no evidence that Kanizsa-type subjective contourscan be detected in parallel'' Perception 25 861 ^ 874

Guzman A, 1969 ` Decomposition of a visual scene into three-dimensional bodies'', in AutomaticInterpretation and Classification of Images Ed.A Grasselli (New York: Academic Press)pp 243 ^ 276

Heitger F, Heydt R von der, 1993 `A computational model of neural contour processing:figure ^ ground segregation and illusory contours'', in Proceedings of the International Confer-ence on Computer Vision volume 4, pp 32 ^ 40

Heydt R von der, Peterhans E, Baumgartner G, 1984 ` Illusory contours and cortical neuronresponses'' Science 224 1260 ^ 1262

Hirsch J, DeLaPaz R L, Relkin N R, Victor J, Kim K, Li T, Borden P, Rubin N, Shapley R, 1995` Illusory contours activate specific regions in human visual cortex: evidence from functionalmagnetic resonance imaging'' Proceedings of the National Academy of Sciences of the USA 926469 ^ 6473

Julesz B, 1964 ` Binocular depth perception without familiarity cues'' Science 145 356 ^ 362Julesz B, 1971 Foundations of Cyclopean Perception (Chicago, IL: University of Chicago Press)Kanizsa G, 1955/1987 ` Margini quasi-percettivi in campi con stimulazione omogenea'' Rivista di

Psicologia 49 7 ^ 30 [translated in Petry S, Meyer G (Eds), 1987 The Perception of IllusoryContours (New York: Springer)]

Kanizsa G, 1979 Organization in Vision (New York: Praeger)Kapadia M K, Ito M, Gilbert C D, Westheimer G, 1995 ` Improvement in visual sensitivity by

changes in local context: parallel studies in human observers and in V1 of alert monkeys''Neuron 15 843 ^ 856

Kellman P, Shipley T, 1991 `A theory of visual interpolation in object perception'' CognitivePsychology 23 141 ^ 221

Kersten D, 1991 ` Transparency and the cooperative computation of scene attributes'', inComputational Models of Visual Processing Eds M S Landy, J A Movshon (Cambridge, MA:MIT Press) pp 209 ^ 228

Koenderink J J, 1990 Solid Shape (Cambridge, MA: MIT Press)Koenderink J J, Doorn A J van, 1976 ` The singularities of the visual mapping'' Biological

Cybernetics 24 51 ^ 59Koffka K, 1935 Principles of Gestalt Psychology (New York: Harcourt Brace)Kojo I, Liinasuo M, Rovamo J, 1993 ` Spatial and temporal properties of illusory figures'' Vision

Research 33 897 ^ 901Kumaran K, Geiger D, Gurvits L, 1996 ` Illusory surface perception and visual organization''

Network: Computation in Neural Systems 7 33 ^ 60Lund J S, 1988 `Anatomical organization of macaque monkey striate visual cortex''Annual Review

of Neuroscience 11 253 ^ 288Malik J, 1987 ` Interpreting line drawings of curved objects'' International Journal of Computer

Vision 1 73 ^ 103Marr D, 1982 Vision (New York: W H Freeman)Mendola J D, Dale A M, Fischl B, Liu A K, Tootell R B, 1999 ` The representation of illusory

and real contours in human cortical visual areas revealed by functional magnetic resonanceimaging'' Journal of Neuroscience 19 8560 ^ 8572

Metelli F, 1974 ` The perception of transparency'' Scientific American 230(4) 90 ^ 98Michotte A, 1963 The Perception of Causality (New York: Basic Books)Minguzzi G F, 1987 `Anomalous figures and the tendency to continuation'', in The Perception of

Illusory Contours Eds S Petry, G Meyer (New York: Springer) pp 71 ^ 75Mumford D, Kosslyn S M, Hillger L A, Herrnstein R J, 1987 ` Discriminating figure from

ground: the role of edge detection and region growing'' Proceedings of the National Academyof Sciences of the USA 84 7354 ^ 7358

Nakayama K, He Z J, Shimojo S, 1995 ` Visual surface representation: a critical link betweenlower-level and higher-level vision'', in Visual Cognition Eds S M Kosslyn, D N Osherson(Cambridge, MA: MIT Press) pp 1 ^ 70

Nakayama K, Shimojo S, 1992 ` Experiencing and perceiving visual surfaces'' Science 257 1357 ^ 1363Peterhans E, Heydt R von der, 1989 ` Mechanisms of contour perception in monkey visual

cortex. II. Contours bridging gaps'' Journal of Neuroscience 9 1749 ^ 1763Peterhans E, Heydt R von der, 1991 ` Subjective contoursöbridging the gap between psychophysics

and physiology'' Trends in Neurosciences 14 112 ^ 119Peterson M A, 1994 ` Object recognition processes can and do operate before figure ^ ground

organization'' Current Directions in Psychological Science 3 105 ^ 111

The role of junctions in surface completion and contour matching 365

Peterson M A, 1999 ` What's in a stage name? Comment on Vecera and O'Reilly (1998)'' Journalof Experimental Psychology: Human Perception and Performance 25 276 ^ 286

Peterson M A, Gibson B S, 1994 ` Must figure ^ ground organization precede object recognition?An assumption in peril'' Psychological Science 5 253 ^ 259

Ratoosh P, 1949 ``On interposition as a cue for the perception of distance'' Proceedings of theNational of the Academy of Sciences of the USA 35 257 ^ 259

Reynolds R I, 1981 ` Perception of an illusory contour as a function of processing time'' Perception10 107 ^ 115

Ringach D L, Shapley R, 1996 ``Spatial and temporal properties of illusory contours and amodalboundary completion'' Vision Research 36 3037 ^ 3050

Rock I, 1983 The Logic of Perception (Cambridge, MA: MIT Press)Rock I, 1987 `A problem-solving approach to illusory contours'', in The Perception of Illusory

Contours Eds S Petry, G Meyer (New York: Springer) pp 62 ^ 70Rubin E, 1921 Visuell wahrgenommene Figuren (Copenhagen: Gyldendals)Rubin N, 1997 `A crucial role for L-junctions and T-junctions in the perception of modal and

amodal completion'' Investigative Ophthalmology & Visual Science 38(4) S991Rubin N, Bauer E, Reitzen S, 1999 ` Scene segmentation occurs in two stages: experimental

evidence and a model'' Investigative Ophthalmology & Visual Science 40(4) S4192Rubin N, Nakayama K, Shapley R, 1997 `Abrupt learning and retinal size specificity in illusory-

contour perception'' Current Biology 7 461 ^ 467Saund E, 1999 ``Perceptual organization of occluding contours of opaque surfaces'', CVIU Special

Issue on Perceptual OrganizationSharon E, Brandt A, Basri R, 2000 ``Fast multiscale image segmentation'', in Proceedings of the

IEEE Conference on ComputerVision and Pattern Recognition, South Carolina (Washington, DC:IEEE Computer Society Press) pp 70 ^ 77

Sheth B R, Sharma J, Rao S, Sur M, 1996 ` Orientation maps of subjective contours in visualcortex'' Science 274 2110 ^ 2115

Shi J, Malik J, 1997 ` Normalized cuts and image segmentation'', in Proceedings of the IEEEConference on Computer Vision and Pattern Recognition, Puerto Rico (Washington, DC:IEEE Computer Society Press) pp 731 ^ 737

Shimojo S, Nakayama K, 1990 `Amodal representation of occluded surfaces: role of invisiblestimuli in apparent motion correspondence'' Perception 19 285 ^ 299

Shimojo S, Silverman G, Nakayama K, 1989 ` Occlusion and the solution to the aperture problemfor motion'' Vision Research 29 619 ^ 626

Shipley T, Kellman P, 1990 ` The role of discontinuities in the perception of subjective figures''Perception & Psychophysics 48 259 ^ 270

Shomstein S, Rubin N, 1998 ` Searching for an illusory shape: not so fast'' Investigative Ophthal-mology & Visual Science 39(4) S789

Singh M, Hoffman D, 1998 ` Completing visual contours: The relationship between relatabilityand minimizing inflections'', Preprint MBS 98 ^ 04

Sugita Y, 1999 ` Grouping of image fragments in primary visual cortex'' Nature 401 269 ^ 272Takahashi S, 1991 ` Operation of figural cues in the microgenetic process of subjective contour

perception'' Japanese Journal of Psychology 62 212 ^ 215Tse P, Albert M, 1998 `Amodal completion in the absence of image tangent discontinuities''

Perception 27 455 ^ 464Ullman S, 1976 ` Filling-in the gaps: the shape of subjective contours and a model for their

generation'' Biological Cybernetics 25 1 ^ 6Ullman S, 1995 ` Sequence seeking and counter streams: A computational model for bidirectional

information flow in the visual cortex'' Cerebral Cortex 5 1 ^ 11Wallach H, Slaughter V, 1988 ` The role of memory in perceiving subjective contours'' Perception

& Psychophysics 43 101 ^ 106Williams L R, Jacobs D W, 1997 ` Stochastic completion fields: a neural model of illusory contour

shape and salience'' Neural Computation 9 837 ^ 858Wouterlood D, Boselie F, 1992 `A good continuation model of some occlusion phenomena''

Psychological Research/Psychologische Forschung 54 267 ^ 277Yen S C, Finkel L H, 1998 ` Extraction of perceptually salient contours by striate cortical networks''

Vision Research 38 719 ^ 741Yin C, Kellman P J, Shipley T F, 1997 ` Surface completion complements boundary interpolation

in the visual integration of partly occluded objects'' Perception 26 1459 ^ 1479

ß 2001 a Pion publication printed in Great Britain

366 N Rubin