Perception and Misperception of Surface Opacity

Perception and misperception of surface opacity

Phillip J. Marlowa,1, Juno Kimb, and Barton L. Andersona,1

aSchool of Psychology, University of Sydney, NSW 2006, Sydney, Australia; and bSchool of Optometry and Vision Science, NSW 2052, Sydney, Australia

Edited by Wilson S. Geisler, University of Texas at Austin, Austin, TX, and approved November 7, 2017 (received for review June 29, 2017)

A fundamental problem in extracting scene structure is distinguish- be computable from the images because it requires information ing different physical sources of image structure. Light reflected by about surface geometry that is not in view. The complexity of light an opaque surface covaries with local surface orientation, whereas interactions with translucent materials has motivated the sugges- light transported through the body of a translucent material does tion that the perception of translucency may depend entirely on not. This suggests the possibility that the visual system may use the heuristic “rules of thumb,” similar to those employed by artists to covariation of local surface orientation and intensity as a cue to the depict such attributes (16). opacity of surfaces. We tested this hypothesis by manipulating the There may, however, be some general generative constraints that contrast of luminance gradients and the surface geometries to which the visual system may use to compute the opacity of surfaces. One they belonged and assessed how these manipulations affected the characteristic feature of opaque surfaces is that the intensity of perception of surface opacity/translucency. We show that (i) identi- light projected to the image covaries with surface orientation in cal luminance gradients can appear either translucent or opaque light fields with a dominant illumination direction, which generates depending on the relationship between luminance and perceived patterns of surface shading. The intensity of diffuse reflectance 3D surface orientation, (ii) illusory percepts of translucency can be declines with increasing angle between the surface normal and illumination direction. Although this covariation can be reduced by induced by embedding opaque surfaces in diffuse light fields that cast shadows and interreflections, the covariation of intensity and eliminate the covariation between surface orientation and intensity, iii surface orientation will not be completely eliminated, particularly and ( ) illusory percepts of opacity can be generated when trans- for locally convex surface patches. In contradistinction, the sub- parent materials are embedded in a light field that generates images surface scattering that arises with translucent materials can pro- where surface orientation and intensity covary. Our results provide duce smooth image gradients that completely eliminate the insight into how the visual system distinguishes opaque surfaces and systematic covariation of intensity and surface orientation. This light-permeable materials and why discrepancies arise between the suggests that the covariation of intensity and surface orienta- perception and physics of opacity and translucency. These results tion could provide information that the visual system uses to suggest that the most significant information used to compute the evaluate the opacity of surfaces: If intensity covaries smoothly perceived opacity and translucency of surfaces arise at a level of with local surface orientation, it provides evidence that this representation where 3D shape is made explicit. light has been reflected by the surface (i.e., shading); however, if intensity varies smoothly but independently of local surface visual perception | 3D shape | material perception | reflectance | translucency orientation, then it could provide information that the light projected to a vantage point is generated by light–material in- fundamental goal of vision research is to understand how teractions that occur within the body of an object. Athe brain derives the appearance of objects and materials One of the primary assumptions shaping the work described from image structure. This problem is difficult because image herein is that surface opacity constitutes a physical and percep- structure arises from a number of distinct physical sources: 3D tual dimension, rather than distinct categories of materials. The shape, reflectance, transmittance and/or subsurface scattering, same assumption is embodied in models of transparency. The endpoints of this dimension can be illustrated by plotting intensity and the light field. Our ability to perceive the color, lightness, as a function of surface orientation for an opaque and translu- gloss, translucency, and shape of materials implies that there cent material with the same 3D shape (Fig. 1A). In this example, must be information available to perform these computations, the translucent surface is illuminated by a punctate source although such processes are just beginning to be understood. Recent work on opaque surfaces has shown that the perception of reflectance depends on both physical (1–8) and perceived (9– Significance 12) 3D shape. The 3D shape of surfaces modulates the pattern of specular reflections projected to the eyes, which induces corre- One of the most perplexing problems in vision science is to un- sponding variations in the perception of gloss (1–8). Perceived 3D derstand how the visual system computes the reflectance and shape can also alter the apparent reflectance of identical lumi- transmittance properties of different materials despite variability in nance gradients (9–12). When a fixed image gradient appears to 3D shape and illumination. We show that 3D shape can play a be generated by a shaded surface with a high curvature, it appears decisive role in the perception of surface opacity and that gross more matte than when the same gradient appears to be generated misperceptions of surface opacity can arise for specific surface by a lower surface curvature (10–12). These results suggest that geometries and light fields. We argue that the perception of sur- the reflectance properties of surfaces are derived at a level of face opacity depends on the relationship between intensity and 3D representation where 3D shape is made explicit. surface orientation and show that this relationship can play a de- The physics characterizing the interaction of light and trans- cisive role in modulating the perception of surface opacity, in- lucent materials is also affected by its 3D shape and is more dependently of the true physical characteristics of materials. complicated than the physics of surface reflectance. The light projected from a translucent material depends on its 3D surface Author contributions: P.J.M., J.K., and B.L.A. designed research; P.J.M. and B.L.A. per- geometry, reflectance properties, and how light is transported formed research; P.J.M. analyzed data; and P.J.M. and B.L.A. wrote the paper. into and out of its volume. The amount, spectral content, and The authors declare no conflict of interest. direction of the light crossing the surface of the material depend This article is a PNAS Direct Submission. on its refractive indices, the viewing direction, and angle of in- Published under the PNAS license. cident light. The light scattered within a material depends on 1To whom correspondence may be addressed. Email: [email protected] or the density and reflectance of the particles suspended within its [email protected]. – body (13 15). The amount of light transmitted through a This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. translucent object also depends on its thickness, which may not 1073/pnas.1711416115/-/DCSupplemental.

13840–13845 | PNAS | December 26, 2017 | vol. 114 | no. 52 www.pnas.org/cgi/doi/10.1073/pnas.1711416115 Downloaded by guest on October 3, 2021 of the rendered surface. The physically translucent object had a A opaque translucent shape resembling a bloated “snake” that varied in thickness along its medial axis. It was illuminated from behind (see Methods), which resulted in intensity maxima at the narrower regions and intensity minima at the thicker regions. The projected intensities of the gradients generated by the translucent snake do not exhibit any systematic covariation with surface orientation (Fig. 2B, Upper Left). A second shape—atwisted“ribbon”—was constructed so that the intensities generated by the snake would be mapped onto a surface such that they covaried with local surface orientation (Fig. 2B, Upper Right). If the visual system uses this covariation to infer the opacity of a surface, the ribbon should appear opaque even though the image B gradients were generated by an object that was translucent. opaque surface normal translucent The second pair of surfaces was created by first generating a 1.0 1.0 surface where intensity varies perfectly with surface orientation (a 0.8 0.8 “bump”;Fig.2B, Lower Right) and then mapping these gradients 0.6 0.6 onto a second shape that eliminated this covariation (a “torus”;Fig.

0.4 0.4 2B, Lower Left). Note that neither of these surfaces/materials was

radiance rendered; the gradients were texture mapped onto each of the two 0.2 0.2 surfaces to create the stimuli. The surfaces for both pairs of surfaces 0.0 0.0 0 30 60 90 120 150 180 0 306090120150180 were covered in a black dot or grid texture superimposed over the orientation orientation luminance gradients, and the two 3D shape percepts were generating using binocular disparity or structure from motion. The contrast of Fig. 1. A shows an opaque surface illuminated mainly from above and a the luminance gradients of the two shapes was varied in seven steps translucent material illuminated internally by a punctate source located inside by multiplicatively scaling the range of image intensities. Observers the body of the material. B plots the image intensities as a function of the angle performed paired comparison experiments in which each combina- in degrees between the surface normal and illumination directions (see Methods). tion of the two 3D shapes and seven different values of image contrast were presented, and observers selected the surface that appeared more translucent in each pair. Different groups of naïve located inside the object, whereas the opaque (Lambertian) surface ≥ is illuminated externally by four rectangular area lamps. Local sur- observers (n 10 in each experiment; see Methods)wereusedfor face orientation is depicted by a circular base tangent to the surface each pair of 3D shapes (snake versus ribbon or bump versus torus) and a line depicting the outward pointing surface normal (Fig. 1B). and for each shape cue (binocular or structure from motion). The intensity of each pixel is plotted as a function of the angular The effect of perceived shape on perceived material can be separation (slant) of surface normals with respect to the illumination directly experienced by the reader by viewing Fig. 3 and Movies S1 direction that generates the strongest intensity–orientation co- and S2. Movies S1 and S2 are example stimuli used in the ex- variation (see Methods). Note that the translucent object exhibits no periments shown in Fig. 2. Example stimuli for the stereoscopic COGNITIVE SCIENCES clear dependence of intensity on surface orientation. In contradis- experiments are shown in Fig. 3. The shape interpretations where PSYCHOLOGICAL AND tinction, the opaque surface exhibits a low-dimensional (∼1D) de- surface orientation and intensity exhibit a low-dimensional co- pendence of intensity on surface orientation. The specific shape of variation appear opaque (the bump and ribbon), whereas the this “contour” depends primarily on the diffuse and specular re- shape interpretations that do not exhibit this covariation appear flectance of the surface as well as the locations of shadows. In this translucent (the torus and snake). Note also that the perception of (Lambertian) example, intensity declines as a cosine function of the illumination also differs for these two shapes. The shapes that angular difference between the primary illumination direction until appear translucent appear to transmit light from a source located ∼90° from the dominant overhead light source; orientations greater behind or inside the object, whereas the shapes that appear opa- than 90° generate an attached shadow, and the intensities of these que appear to reflect light from above. The results of experiment regions depend on illumination from the illuminants from below and 1areshowninFig.2A, which plots the proportion of trials that to its sides. The “thickness” or spread along the contour depends on each stimulus was chosen as appearing more translucent than the factors that weaken intensity–orientation covariation (here, the dif- other surfaces. The contrast of the luminance gradients increases ferent directions and intensities of the multiple light sources). from left to right in each graph. The large, vertical separation The experiments described below were designed to assess whether between the white and black data points demonstrates that ob- the covariation of intensity with 3D surface orientation modu- servers are sensitive to the relationship between intensity and lates the perception of surface opacity. Most previous studies (13– apparent surface orientation when judging translucency: The 21) into the perception of translucency have attempted to identify snake and torus appear more translucent than the ribbon and cues that provide information about the relative opacity/translucency bump. Main and interaction effects of 3D shape and contrast were of a material by holding 3D shape fixed and either identifying or tested using planned within-subject contrasts. The effect of shape manipulating image properties that modulate perceived surface on perceived translucency is statistically significant for both sets of opacity (such as contrast). Here, we take the complementary ap- stimuli [counterclockwise from Fig. 2 A, Top Left: F(1, 13) = 60; proach and fix the luminance gradients in the images and vary the F(1, 13) = 18; F(1, 9) = 255; F(1, 10) = 18, P << 0.01, for each]. The 3D shapes that appear to generate them. Thus, any differences in 3D shape interpretation also influences the effect of image contrast perceived translucency that arise from modulating 3D shape cannot on perceived translucency, producing a significant interaction be- be explained by differences in 2D image properties but must arise at tween the main effects of shape and contrast [counterclockwise a level of representation where 3D shape is made explicit. from Fig. 2 A, Top Left: F(1, 9) = 12, P < 0.01; F(1, 10) = 12, P < 0.01; F(1, 13) = 60, P << 0.01; F(1, 13) = 18, P << 0.01]. Results Decreasing contrast increases perceived translucency for the The Computation of Opacity and Translucency from 3D Intensity Gradients. bump surface, consistent with the view that low contrast is a cue In experiment 1, we rendered images of a translucent and to translucency (18–20). However, the torus, snake, and ribbon opaque surface and generated an alternate shape interpretation shapes appear most translucent for moderate contrast values. for each. The alternate shape was designed to change the relation- This interaction suggests that 3D shape can change the way that ship between image intensity and surface orientation relative to that image contrast influences the perception of surface opacity.

Marlow et al. PNAS | December 26, 2017 | vol. 114 | no. 52 | 13841 Downloaded by guest on October 3, 2021 A surface orientation and intensity. If the visual system interprets the 1.00 Motion Disparity inconsistency between surface orientation and intensity as a cue to translucency, then an opaque surface containing a distribution of 0.75 3D shape convexities and deep concavities should appear translucent when snake rendered in a diffuse illumination. To test this prediction, Lamber- 0.50 ribbon tian and translucent variants of a bumpy surface [similar to that of 0.25 Kim et al. (22)] were rendered in either a Ganzfeld or natural illumination. Observers viewed the surfaces along the direction of sur- 0.00 n=14 n=14 face relief in structure-from-motion displays similar to our previous experiments (Movies S3 and S4). Observers matched the material appearance of each surface by adjusting the material parameters of a 1.00 Motion Disparity cube rendered in a natural light field (Fig. 4). In one experiment, the cube was rendered without specular reflections, and observers only 0.75 adjusted the density of subsurface scattering (Fig. S1A). In a second torus 0.50 experiment, the cube was rendered with specular reflections, and

Proportion chosen more translucent bump observers adjusted both the density of subsurface scattering and in- 0.25 tensity of specular reflections (Fig. S1B). The results of both experiments show the Lambertian surface rendered in a Ganzfeld 0.00 n=14 n=14 appears translucent. The vertical axis represents the density of subsurface scattering particles; the correct match for a Lambertian 0123456 123456 surface is the highest density on the graph (256); the graphical set- Luminance contrast tings for each surface is depicted in Fig. S1D. The results show that I log ( max ) the Lambertian surface rendered in the natural light field appears Imin opaque and matte, whereas the match for the same surface in the Ganzfeld is very translucent and glossy [F(1, 4) = 1,005, P << 0.01, in 1.0 B snake ribbon Fig. S1A; F(1, 4) = 576, P << 0.01, in Fig. S1B;andF(1, 4) = 1,931, 0.8 P << 0.01, for the specular parameter in Fig. S1B]. This result implies that the visual system misattributes the diffuseness of the light 0.6 field to the subsurface scattering of a translucent material in the absence of any covariation in surface orientation and intensity, 0.4 presumably because such materials occur more generically in natural 0.2 scenes than completely diffuse light fields. Our hypothesis also predicts that translucent materials should 0.0 fail to appear translucent if the light transported through the 1.0 material accidentally preserves the covariation of intensity and torus bump

Radiance surface orientation. Fig. 5 depicts two shapes designed to test this 0.8 hypothesis, which are variants of the translucent snake from Fig. 3. In one shape, the diameter of the circular cross-section varies along 0.6 the snake’s medial axis; for the other, the diameter of the cross- 0.4 section remains constant and appears as a curved cylinder. For specific combinations of illumination and subsurface scattering 0.2 density, the intensity of the light transported through the cylinder covaries with surface orientation. Both shapes were rendered as 0.0 identical translucent materials and were embedded in a light field 0 30 60 90 120 150 180 0 30 60 90 120 150 180 with a predominant direction of illumination from behind (see Orientation Methods). Seven different levels of physical translucency were rendered by varying the density of subsurface scattering particles. Fig. 2. The perception of translucency depends on luminance contrast and Lambertian variants of these objects were also rendered within apparent 3D shape. (A) Translucency judgments for the two possible 3D shape the same light field rotated 180° so the predominant direction of interpretations of each intensity gradient pattern. Error bars are SEMs of all observers. The contrast of the gradient increases from left to right in each graph, and illumination was along the line of sight. Seven albedos of the Imax and Imin refer to the maximum and minimum intensity. B plots the re- Lambertian surfaces were chosen to produce the same maximal lationship between intensity and surface orientation for each shape. Orientation is luminance as that generated by the translucent surfaces. Nine ob- the angle in degrees between the surface normal and illumination direction. servers ranked the surfaces from most to least translucent in a paired comparison experiment. Fig. S3B shows the proportion of trials that each surface was chosen as appearing more translucent The Misperception of Opacity. The intensity of an opaque surface will than the other surfaces as a function of the maximal intensity in the only covary with surface orientation in light fields that contain a image. The open symbols depict the translucent renders, and the predominant illumination direction. It is the variation in the 3D pose small gray symbols depict the Lambertian renders. The results of a surface with respect to this illumination direction that is re- reveal a large discrepancy between perceived and physical trans- sponsible for the coupling between surface orientation and intensity lucency for the translucent cylinder. The cylinder appears less of shaded surfaces. Purely diffuse illumination (i.e., illumination in a translucent than the snake even though they are physically the Ganzfeld) eliminates intensity gradients due to shading and hence same material for all values of subsurface density, F(1, 8) = 30, P < eliminates this source of covariation. The intensity variations of a 0.01. Moreover, perceived translucency varies with the density of uniform albedo surface embedded in a purely diffuse light field will subsurface particles differently for the snake and cylinder, F(1, 8) = depend on the pattern of interreflections and the area of the 18.7, P < 0.01. The perceived translucency of the snake increases as Ganzfeld that illuminates each point on a surface (“vignetting”). This density declines (and is hence physically more translucent), whereas area is smaller for deep concavities due to self-occlusion than pro- the cylinder appears less translucent as physical translucency in- truding convexities, which are fully exposed to ambient lighting. creases. The misperception of translucency for the cylinder is well Since interreflections and vignetting depend more on the direction predicted by the emergence of a spurious relationship between and magnitudes of surface curvature than surface orientation, opa- surface orientation and the intensity of light passing through the que surfaces in a Ganzfeld will exhibit inconsistencies between translucent volume (Fig. S3C). These results provide further support

13842 | www.pnas.org/cgi/doi/10.1073/pnas.1711416115 Marlow et al. Downloaded by guest on October 3, 2021 (ii) illusory percepts of translucency can be induced by embedding A Snake Snake opaque surfaces in diffuse light fields that eliminate the covariation between surface orientation and intensity, and (iii) illusory percepts of opacity can be generated when transparent materials are illuminated in a manner that generates images where surface orientation and intensity covary. Thus, opaque materials can appear translucent and translucent materials can appear opaque when they are embedded in light fields that either eliminate or generate systematic covariation in 3D surface orientation and intensity, respectively. Ribbon Ribbon Our hypothesis can also provide insight into illusory percepts of translucency induced by depth map textures (18) and ramp shading (23), where image brightness is determined by the dis- tance of the surface relative to some reference point. These illusory percepts of translucency also appear to depend on the relationship between intensity and local surface orientation. Images that exhibit stronger inconsistencies between intensity and surface normals appear more translucent (the left object in Fig. S4) than those that generate spurious correlations between B Torus Torus intensity and surface normals (the sphere in Fig. S4). The general insight shaping our theoretical approach is that the computation of surface opacity is derived by the way that light covaries with 3D surface geometry. The hypothesis explored herein only considers how intensity varies as a function of local 3D surface orientation because this is the primary generative source of intensity variations generated by opaque surfaces in a directional light field (“shading”). Our hypothesis does not exclude the possibility that higher order 3D shape variables may also be used to compute the opacity/translucency of surfaces or may be used to weight the extent to which the surface orientation/intensity covariation cue is used in such computations. Deep surface concavities can generate shadows, Bump Bump vignetting, and interreflections, all of which disrupt the relationship between surface orientation and intensity (23, 24). Surface convexities, by contrast, are much less subject to these influences and will generally preserve the surface orientation/intensity covariation characteristic of opaque surfaces. This suggests that the pattern of intensity variations across a surface may not all be weighted equally in the visual system’s computation of surface opacity. This can be seen in Fig. S2, which depicts surface normal-intensity plots for a COGNITIVE SCIENCES bumpy Lambertian surface in natural illumination. Despite the PSYCHOLOGICAL AND presence of extensive cast shadows and interreflections present in many regions of the surface, intensity covaries with surface normals on locally convex surface patches where cast shadows and interreflections are less likely. In contradistinction, this covariation is not Right eye Left eye observed in the same regions for the variants of the surface in Fig. Fig. 3. A and B show two possible stereoscopic shape interpretations for S2 that appear translucent (i.e., that were either rendered as the same luminance gradients. The upper shape interpretation in each panel (i.e., the snake and torus) appears as a translucent volume illuminated from within, whereas the lower shape interpretation (ribbon and bump) appears as an opaque surface reflecting light from above. The two eyes’ views are arranged for crossed free fusion.

for the hypothesis that the perception of translucency requires inconsistencies between intensity and 3D surface orientation. Discussion The goal of the work presented herein was to gain insight into how the visual system distinguishes luminance gradients generated by surface shading from gradients generated by subsurface scattering. This problem is computationally difficult because shading and subsurface scattering can generate similar luminance gradients if illumination and 3D shape are free to vary. We hypothesized that the covariation of intensity with surface orientation is a source of information that links projected light to 3D surface geometry and therefore can be used to distinguish image structure generated by optical interactions with surfaces from optical structure generated by Fig. 4. Misperception of surface opacity in diffuse illumination. Two images of subsurface scattering. In support of this hypothesis, we showed that the same bumpy, opaque (lambertian) surface rendered in either a natural light (i) identical luminance gradients can appear either as light trans- field (Left) or diffuse illumination (Right). The diffuse illumination elicits an il- mitted through a light-permeable material or as light reflected from lusory percept of gloss and translucency. The cubes show the appearance of the an opaque surface depending on perceived 3D surface geometry, match surface at PSE in subsurface scattering and specular reflectance (Fig. S1).

Marlow et al. PNAS | December 26, 2017 | vol. 114 | no. 52 | 13843 Downloaded by guest on October 3, 2021 Cylinder - Density 24 Cylinder - Density 24 stimulus also appear translucent. Thus, the experiments presented herein show that 3D shape plays a decisive role in de- termining whether a surface appears translucent or opaque. A number of studies have suggested that image contrast is a significant cue to subsurface scattering, which tends to be reduced by translucent materials by softening shading gradients and shadows (18–20). Our results also show that contrast modulates perceived translucency but revealed that the effects of contrast depended on the particular shape used: The bump appeared most translucent at low contrast, whereas the effect of contrast on the torus was nonmonotonic, appearing most translucent for midlevel values of contrast. To assess whether this result was specific to the contrast manipulation we performed in Snake - Density 24 Snake - Density 24 our experiments (multiplicative scaling), we performed additional experiments to assess a variety of nonlinear contrast transforma- tions, similar to those found to be effective in eliciting percepts of translucency in previous work (18). We observed the same general pattern of results for all forms of contrast tested; the effect of 3D shape had a larger influence on perceived surface opacity than contrast (Fig. S5). Previous work has also shown that the perception of translucency can be significantly enhanced by the addition of specular reflections (18, 19). The luminance gradients in our experiments were texture mapped onto different shapes and therefore did not Right eye Left eye contain specular reflections. We conducted an additional experiment using the same two sets of shape stimuli used in experiment Density of sub-surface particles 1(Movies S1 and S2) and rendered these surfaces with a specular 72 680 463 224 1 component of reflection (Fig. S6 and Movies S5 and S6). We again 1.00 observed large effects of 3D shape on perceived opacity and Shape comparatively smaller effects of contrast. Our results therefore 0.75 Snake Cylinder generalize to stimuli with or without specular reflections. There is continuing debate about the level of representation Translucent 0.50 where material properties are computed. Whereas previous work Opaque

Material has emphasized properties that can be directly computed from 0.25 images, we have argued that the computation of opacity/trans-

Prop. more translucent 0.00 n=9 lucency occurs at a level where 3D shape of surfaces is made ex- – 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 plicit (10 12). This argument is grounded on two assumptions: Intensity (i) The concept of a surface entails some notion of shape (for- mally and perceptually), which in our environment is 3D; and Fig. 5. Misperception of translucency; a physically translucent surface illumi- (ii) opacity/translucency is a property of surfaces. If this is correct, nated from behind that appears opaque and illuminated from the front. The then it follows that the computation of surface opacity must arise images are stereograms arranged for crossed free fusion and depict two objects at a level where surfaces are represented, which entails a repre- that are physically the same translucent material and are rendered in the same sentation of (3D) shape. In support of this view, we have shown illumination (primarily from behind, i.e., back lit). Note that the lower shape that 3D shape can play a decisive role in the perception of opacity/ appears translucent whereas the upper shape is misperceived to be opaque and translucency. Moreover, all previous research into the perception frontally illuminated. The graph shows the results of a paired comparison ex- of surface opacity/translucency used stimuli that evoked vivid periment judging translucency. Error bars are SEMs of all observers. See also Fig. S3. percepts of 3D shape. Indeed, if our logic is correct, it should not be possible to generate a percept of opacity or translucency without a percept of 3D shape. This does not imply that image translucent or rendered as opaque in a diffuse light field). A properties (or statistics) do not provide inputs needed for these promising line of future work will be to assess how well material computations; it simply implies that such inputs do not acquire properties are predicted by different regions of surfaces character- perceptual meaning in specifying the opacity/translucency of a ized by higher order shape descriptors (such as convexities, con- – material until they are linked to a representation of 3D shape. cavities, or saddles), in conjunction with the surface orientation There are a number of outstanding issues raised by the studies intensity analysis proposed herein. – – reported herein. First, our orientation intensity plots were de- Previous studies (13 21) that have investigated cues to per- rived using the ground truth surface orientations and intensities ceived translucency have held 3D shape fixed and either varied rather than any direct measurements of perceived shape. Al- the amount of simulated translucency (subsurface scattering) or though the orientation/intensity computations mush be derived directly manipulated image properties in an attempt to modulate from perceived 3D shape, any distortions in perceived shape perceived opacity/translucency. Here, we took the complemen- relative to ground truth in our stimuli would likely be related by a tary approach and held image properties fixed and varied 3D monotonic transformation of surface orientation and illumina- shape. A related approach was taken by Kim et al. (22), who tion direction [e.g., affine distortions (25, 26)], and hence do not reported that the percept of “glow” is also coupled to repre- affect our main arguments. Second, although vivid percepts of sentations of 3D shape. They constructed bumpy surfaces ren- 3D shape and opacity/translucency can arise from static mon- dered in a diffuse light field similar to those depicted in Fig. 4 ocular images, there is currently no explanation of how 3D shape and found that a percept of glow could be induced by inverting can be computed from such images. The stimuli and analyses the depth of the peaks and valleys while holding the intensity described herein exploited parallax cues to elicit a particular 3D gradients fixed. They mentioned a possible link between trans- shape percept, and hence offer no insight into how the visual lucency and their (inverted) glowing stimulus but did not assess system solves this problem. Lastly, our experiments—like all perceived translucency in either of their stimuli. The experiments previous studies into the perception of opacity/translucency— reported herein demonstrate that variants of their noninverted have only investigated uniform albedo surfaces, so it is unclear

13844 | www.pnas.org/cgi/doi/10.1073/pnas.1711416115 Marlow et al. Downloaded by guest on October 3, 2021 whether our results generalize to stimuli containing densely varying more complete understanding of precisely how computations of surface albedo. We therefore performed additional experiments 3D shape and material properties are coupled, i.e., how the vi- using stimuli containing variations is surface pigmentation, and sual system uses 3D shape to derive material properties, and how observed the same dramatic effect of 3D shape on perceived sur- material properties influence the computation of 3D shape. face opacity/translucency (Fig. S7). It is clear from viewing these stimuli that the visual system successfully segments the contribu- Methods tions of texture from the smooth gradients generated by shading or The experiments were approved by the University of Sydney and participants subsurface scattering, although it remains unknown how the visual provided written informed consent and were debriefed about the aims in system performs this decomposition. Thus, the same orientation/ adherence with the Declaration of Helsinki. The three inputs to the surface intensity analysis proposed herein can account for these textured orientation-intensity analysis were a RADIANCE high dynamic range (HDR) stimuli under the proviso that the intensity variations caused by image of the surface, its depth map, and a list of illumination directions to be albedo (texture) are correctly identified as texture. analyzed, which covered the spherical space of possible illumination direc- The results presented herein and elsewhere indicate that the tions. For each illumination direction, the image intensities were plotted perception of surface opacity/translucency, like the perception of against the angular separation of the surface normals relative to that di- specular and diffuse reflectance, depends on the apparent 3D rection. The plot shown for each surface was selected because it exhibits the structure of images (9–12, 23, 27–29). Indeed, we have argued strongest negative correlation between intensity and surface orientation of the concept of a surface, both conceptually and perceptually, all of the illumination directions analyzed. See Fig. S8 and Supporting In- entails a notion of shape, which in our environment is 3D. If this formation for the full methods. is correct, it suggests that the representations used to derive properties such as opacity/translucency occur at a level where 3D ACKNOWLEDGMENTS. This research was supported by grants awarded (to shape is made explicit. Future research is required to develop a B.L.A.) from the Australian Research Council.

1. Ho YX, Landy MS, Maloney LT (2008) Conjoint measurement of gloss and surface 16. Koenderink J, van Doorn A (2001) Shading in the case of translucent objects. texture. Psychol Sci 19:196–204. Proceeding of the SPIE Conference on Human Vision and Electronic Imaging, 10.1117/ 2. Olkkonen M, Brainard DH (2011) Joint effects of illumination geometry and object 12.429502. shape in the perception of surface reflectance. Iperception 2:1014–1034. 17. Singh M, Anderson BL (2002) Perceptual assignment of opacity to translucent sur- 3. Marlow PJ, Kim J, Anderson BL (2012) The perception and misperception of specular faces: The role of image blur. Perception 31:531–552. surface reflectance. Curr Biol 22:1909–1913. 18. Fleming RW, Bülthoff HH (2005) Low-level image cues in the perception of trans- 4. Marlow PJ, Anderson BL (2013) Generative constraints on image cues for perceived lucent materials. ACM Trans Appl Percept 2:346–382. gloss. JVis13:2. 19. Motoyoshi I (2010) Highlight-shading relationship as a cue for the perception of 5. Qi L, Chantler MJ, Siebert JP, Dong J (2014) Why do rough surfaces appear glossy? translucent and transparent materials. JVis10:6. J Opt Soc Am A Opt Image Sci Vis 31:935–943. 20. Nagai T, et al. (2013) Image regions contributing to perceptual translucency: A psy- 6. Nishida S, Shinya M (1998) Use of image-based information in judgments of surface- chophysical reverse-correlation study. Iperception 4:407–428. reflectance properties. J Opt Soc Am A Opt Image Sci Vis 15:2951–2965. 21. Kahn EA, Reinhard E, Fleming RW, Bulthoff HH (2006) Image-based material editing. 7. Wendt G, Faul F, Ekroll V, Mausfeld R (2010) Disparity, motion, and color information ACM Trans Graph 25:654–663. improve gloss constancy performance. JVis10:7. 22. Kim M, Wilcox LM, Murray RF (2016) Perceived three-dimensional shape toggles 8. Wijntjes MW, Pont SC (2010) Illusory gloss on Lambertian surfaces. JVis10:13. perceived glow. Curr Biol 26:R350–R351. 9. Todorovic D (2014) How shape from contours affects shape from shading. Vision Res 23. Todd JT, Egan EJ, Phillips F (2014) Is the perception of 3D shape from shading based 103:1–10. on assumed reflectance and illumination? Iperception 5:497–514. 10. Marlow PJ, Todorovic D, Anderson BL (2015) Coupled computations of 3D shape and 24. Todd JT, Egan EJ, Kallie CS (2015) The darker-is-deeper heuristic for the perception of COGNITIVE SCIENCES material. Curr Biol 25:R221–R221. 3D shape from shading: Is it perceptually or ecologically valid? JVis15:2. PSYCHOLOGICAL AND 11. Marlow PJ, Anderson BL (2015) Material properties derived from three-dimensional 25. Kunsberg B, Zucker SW (2014) How shading constrains surface patches without shape representations. Vision Res 115:199–208. knowledge of light sources. SIAM J Imaging Sci 7:641–668. 12. Marlow PJ, Anderson BL (2016) Motion and texture shape cues modulate perceived 26. Belhumeur P, Kriegman D, Yuille A (2004) The bas-relief ambiguity. Int J Comput Vis material properties. JVis16:5. 35:33–44. 13. Gkioulekas I, et al. (2013) Understanding the role of phase function in translucent 27. Gilchrist AL (1977) Perceived lightness depends on perceived spatial arrangement. appearance. ACM Trans Graph 32:147. Science 195:185–187. 14. Gkioulekas I, Walter B, Adelson EH, Bala K, Zickler T (2015) On the appearance of translucent 28. Knill DC, Kersten D (1991) Apparent surface curvature affects lightness perception. edges. IEEE Conference on Vision and Pattern Recognition, 10.1109/CVPR.2015.7299192. Nature 351:228–230. 15. Xiao B, et al. (2014) Looking against the light: How perception of translucency de- 29. Bloj MG, Kersten D, Hurlbert AC (1999) Perception of three-dimensional shape in- pends on lighting direction. JVis14:17. fluences colour perception through mutual illumination. Nature 402:877–879.

Marlow et al. PNAS | December 26, 2017 | vol. 114 | no. 52 | 13845 Downloaded by guest on October 3, 2021