Journal of Vision (2014) 14(3):17, 1–22 http://www.journalofvision.org/content/14/3/17 1 Looking against the light: How perception of translucency depends on lighting direction Department of Computer Science, American University, Bei Xiao Washington, DC, USA # $ Department of Computer Science, Cornell University, Ithaca, Bruce Walter NY, USA # $ Harvard School of Engineering and Applied Sciences, Ioannis Gkioulekas Cambridge, MA, USA # $ Harvard School of Engineering and Applied Sciences, Todd Zickler Cambridge, MA, USA # $ Department of Brain and Cognitive Sciences, Edward Adelson Massachusetts Institute of Technology, Cambridge, MA, USA Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA Department of Computer Science, Cornell University, Kavita Bala Ithaca, NY, USA # $ Translucency is an important aspect of material (2013) to span an important range of translucent appearance. To some extent, humans are able to appearance. We find the degree of translucency estimate translucency in a consistent way across constancy depends strongly on the phase function’s different shapes and lighting conditions, i.e., to exhibit location in the same 2D space, suggesting that the space translucency constancy. However, Fleming and Bulthoff¨ captures useful information about different types of (2005) have shown that that there can be large failures translucency. We also find that the geometry of an object of constancy, with lighting direction playing an important is important. We compare the case of a torus, which has role. In this paper, we explore the interaction of shape, a simple smooth shape, with that of the figurine, which illumination, and degree of translucency constancy more has more complex geometric features. The complex deeply by including in our analysis the variations in shape shows a greater range of apparent translucencies translucent appearance that are induced by the shape of and a higher degree of constancy failure. In summary, the scattering phase function. This is an aspect of humans show significant failures of translucency translucency that has been largely neglected. We used constancy across changes in lighting direction, but the appearance matching to measure how perceived effect depends both on the shape complexity and the translucency depends on both lighting and phase translucency phase function. function. The stimuli were rendered scenes that contained a figurine and the lighting direction was represented by spherical harmonic basis function. Observers adjusted the density of a figurine under one lighting condition to match the material property of a Introduction target figurine under another lighting condition. Across the trials, we varied both the lighting direction and the Many natural materials we encounter every day are phase function of the target. The phase functions were translucent, including the food we eat, the liquids we sampled from a 2D space proposed by Gkioulekas et al. drink, and our skin. The translucent appearance of Citation: Xiao, B., Walter, B., Gkioulekas, I., Zickler, T., Adelson, E., & Bala, K. (2014). Looking against the light: how perception of translucency depends on lighting direction. Journal of Vision, 14(3):17, 1–22, http://www.journalofvision.org/content/14/3/17, doi:10.1167/14.3.17. doi: 10.1167/14.3.17 Received October 7, 2013; published March 13, 2014 ISSN 1534-7362 Ó 2014 ARVO Journal of Vision (2014) 14(3):17, 1–22 Xiao et al. 2 Figure 1. Translucent materials. (A) Studio photographs of a cherub figurine made with polyurethane lit from two lighting directions, from behind and from the side. The one on the left looks more translucent. (B) Photographs of two different translucent materials under the same natural illumination. The same ‘‘frog prince’’ shape was made of two different translucent materials, soap, and wax. Observers can discriminate the subtle differences between the two materials. these materials is caused by internal volumetric translucent appearance of the two materials is very scattering. Figure 1A shows photographs of a cherub different. Since in the natural world translucent under two different lighting directions. One distinctive materials are the norm, not the exception (Donner et feature of the material is that it is permeable to light. al., 2009), it makes sense that the human visual system We see part of the green background through the thin is well engineered to analyze them. However, very little parts of the cherub, such as its wings. This is because is known about how the visual system recognizes and some light penetrates into the cherub that allowed the discriminates translucent materials from images. background to be seen through the object. Creating controlled inputs to study translucency Humans are skilled at discriminating subtle differ- with real objects is a challenge as it is hard to ences in translucency, such as when they discriminate manufacture objects with specific scattering parame- milk from cream, wax from soap, and human flesh ters. Further, until very recently rendering realistic from a doll. Figure 1B shows photographs under translucent objects was not feasible using computer natural illumination of the same ‘‘frog prince’’ shape simulation, thus preventing the use of such stimuli for made of two different materials, soap and wax; the controlled studies. Therefore, most previous work in Journal of Vision (2014) 14(3):17, 1–22 Xiao et al. 3 translucency perception has focused on stimuli with Background and related work simple shapes, unnatural illumination, and a limited set of materials. These works hypothesized that image cues We begin by introducing the physical process of such as specular highlights, blur, image contrast, and subsurface scattering that is responsible for the color variation could influence the perception of appearance of the translucent objects, the phase translucency. However, to understand translucency function models, and current rendering methods for perception in the real world, we need to expand on each translucency. We will then review previous studies on of these three parameters: We need to consider complex material perception and constancy, with an emphasis shapes, realistic lighting, and a richer set of materials. on the effects of lighting environment and 3D shape. It is commonly observed that lighting has dramatic effects on translucent objects: They tend to appear more translucent under backlighting. For example, Translucent materials people often hold up a translucent object against a light Subsurface scattering: Figure 2A illustrates how light while studying it. This change in lighting direction also interacts with translucent objects and the parameters causes many of the measureable image properties (such controlling subsurface scattering. When light hits the as image contrast, location of specular highlights, cast surface of an opaque object, it bounces back from the shadows, luminance histograms, etc.) to change. point that is illuminated. When light hits a translucent Studying the effect of lighting direction on translucency object, only some of it bounces off from the surface. can be very informative of the images cues used by the The rest penetrates into the object, refracts, and visual system. In addition, the lighting effect varies scatters multiple times within the body of the medium, depending on the 3D shapes and material properties of until it re-emerges outside from a different location. In objects. For example, the translucent appearance of a computer graphics, this process can be modeled using jade bracelet might be more sensitive to a change of the radiative transfer equation (RTE) (Ishimaru, 1978). lighting direction than a bar of soap. Therefore, This approach models the material using three wave- studying lighting on translucency perception with a length dependent functions, the scattering coefficient rs, controlled 3D shape and material properties is impor- the absorption coefficient ra, and the phase function p. tant. For light at a single wavelength, rs and ra describe how Changing lighting direction also influences the rapidly the light is scattered and absorbed in the perception of 3D shape of an object, which indirectly medium. When light is scattered, the phase function, influences the perception of material properties (Ger- which is a probability distribution over directions, ardin, Kourtzi, & Mamassian, 2010; Koenderink & van describes the angular distribution of scattered light. Doorn, 2001; Olkkonen & Brainard, 2011; Todd, 2004; The sum of the scattering and absorption coefficients is Wijntjes & Pont, 2010). A previous study on the effect called the extinction coefficient, rt rs ra, also referred ¼ þ of lighting direction on translucency used a simple to as the density of the material, and the ratio rs/rt is shape, a torus (Fleming & Bu¨lthoff, 2005). By called the albedo. The density controls the degree of observing translucent objects in real life, we find translucency of the medium. A more translucent lighting direction influences objects that have a medium has a low density while a less translucent variation of thin and thick geometry, such as the medium has a high density. When light interacts with cherub shown in Figure 1A, in a different way from an the medium, the albedo controls what fraction of this object with a simple geometry, such as a torus (Xiao et light is scattered instead of being absorbed. The al., 2012). Gkioulekas et al. (2013) showed that scattered light then continues in new directions