DESIGN Pi2inciples of HARDWARE-BASED PHONG SHADING and BUMP-MAPPING

DESIGN Pi2INCIPLES OF HARDWARE-BASED PHONG SHADING AND BUMP-MAPPING

K. BENNEBROEK’, I. ERNST’, H. RUSSELER2 and 0. WITTIG2+

‘Department of Electrical Engineering,Networking Theory Group, University of Twente, Twente, The Netherlands *German National Research Center for Computer Science,Institute for Computer Architecture and Software Technology (GMD FIRST), Rudower Chaussee 5, 12489Berlin, Germany e-mail: [email protected]

Abstract-The VISA+ hardwarearchitecture is the first of a new generationof graphicsaccelerators designedprimarily to renderbump-, texture-, environment- and environment-bump-mappedpolygons. Thispaper presents examples of the maingraphical capabilities and discusses methods and simplifications usedto createhigh quality images.One of the keyconcepts in the VISA + design,the useof reflectance cubes,is predestined for environmentmapping. In combinationwith bump-and texture-mapping it shows the strengthof our new architecture.Furthermore it justifiessome of the decisionsmade during simulationand developmentof the complexVISA+ architecture.0 1997Elsevier Science Ltd

1. INTBODUCI’ION interpolation approach can be important if diffuse Even the fastest high quality graphics workstations intensitiesare calculatedon an incrementalbasis. like the recently announcedInfiniteReality@ graphics The normal-vector shading approach of the acceleratorsare still basedon common texturing in GMD-FIRST is a reflection map method. This combination with Gouraud shading. With continu- shadingtechnique, implemented in the VISA system ously improving VLSI technologiesin combination [5l operatesin a similar way to the reflection-vector with CAD tools to support full custom chip design, shadinghardware of Voorhies and Foran [6], new graphic architectures must evolve. As an A logical extension of the normal vector inter- important step towards higher realism, real-time polation principle was to support bump-mapping[9]. Phong shading will be the next technology-pushin Bumps improve visual effects for low-end game, as computer graphics. well as high-endVR applications.A combination of Over the past few decades,many attempts have texture and bump-mapsfor extremely perspective been made, to make Phong shadingpracticable for and huge polygons reduces the total setup- and hardware implementation. Bishop and Weimar [1] transfer-timefor computer graphicsscenes. proposed a Taylor seriesapproximation for N.H. In this paper, the quality aspectsof homogeneous For a curvature lessthan 60”, forward differencinga versus non-homogeneousinterpolation for Phong quadratic polynomial fits quite well and reducesthe shading,bump-mapping, environment and environ- computational efforts to five additions per pixel. ment bump-mapping,will be discussed.Focusing on Deering [2] presenteda shaderthat interpolatesthe efficient division schemesfor perspectivetexturing normal and eye vectors. In both approaches, and cube-reflectancemapping, a proposedhardware exponentiation of the cosine function was done by architectureis discussedin more detail. table lookup, which leadsto intolerable hardware size if a broad range of exponentsis to be supported. 2. BASIC FUNDAMENTALS Another method usesangular interpolation tech- 2.1. Phongshading niquesrather than vector interpolation. This has the Phong Illumination was introduced by Tong Bui advantagethat the vector length remainsunchanged Phong in 1975[7] as an empiricalmodel to simulate during the interpolation. This way, normalization highlights on shiny surfaces.The model has been can be reduced to vectors at the vertices and acceptedas a very good approximation of physical calculatedin software. While an approach suggested highlights. Phong introduced a specular exponent, by U. Clausen [3] interpolatespolar angles,Kuijk reflecting materialproperties, and modelledthe steep and Blake [4] expand this principle to angular falloff of the highlight as the cosine of the angle interpolation on great-circles.The resulting expres- between the eye-vector and reflected light-vector, sions,however, are rather complicated.The angular raisedto the specularexponent. For the implementation of Phong Illumination on a per pixel basis,the normalvector has to be interpolated over a polygon. Normal vector interpolation shadersare therefore + Author for correspondence. called Phong shaders,in contrast to the commonly

143 144 K. Bennebroek,1. Ernst,H. Riisselerand 0. Wittig usedGouraud shadersthat interpolate color-values over the polygon (that may have been calculated using Phong Illumination!). Phong’s Illumination model consists of an ambient, a diffuse and a Note the additional BumpMorph parameterwhich specularcomponent, resultingin: is used to control the amplitude of the bump perturbation (length of P). Since this parameter is I=a+~(d(N.L)+s(R.E)“) (1) constant over the polygon, it can be included in the I matrix A. This way, Equation (3) can be simplifiedto: for all lightsourcesi in the scene,with a, d and s ambient, diffuse and specular column-vectors (R,G,B)= respectively to depict material and lightsource properties. N depicts the surface normal To convert Pi from WC to ScreenSpacewe need vector, L the vector pointing from the point on the two multiplications for each of its componentsby i surfaceto the light source,R the mirrored vector of However, in Section 3.2 we explain why we have to L around N (reflected light vector) and E the calculate the reflected ray in WC insteadof Screen- reflected eye vector. All vectors need to be normal- Space coordinates. Hence we must convert the ized. normal N from ScreenSpaceto WC too. This is done by three divisions (i). 2.2. Bump-mapping The resulting formula, which is implemented in In 1974, Blinn [8] developed a technique that our hardware, then becomes: enables a surface to appear wrinkled or dimpled N’=N+[A.P] (5) without the need to model this roughnessgeome- trically. Note that N is generated by interpolating the For this purpose,he defined a specialtexture map normalized edge normal vectors of the triangle. called bump-map B(u,v) containing displacementor Hence N is unnormalized inside the triangle which height values to perturb the local normal vector. The leads to small highlight shifts on the surface. This perturbed normal vector is given asN’= N+ D where artifact can not be recognizedbecause there are no D is the perturbation vector. To calculate D, he color differencesnear the edgesof adjacent triangles definedtwo other vectors lying in the tangent plane [91. of the surfacegiven by N. Thesevectors are defined The alignmentmatrix A can be usednot only to as A=NxO, and B=NxO,, where O(u,v) is a get the correct perturbation vector, but also to parametrized function representing the position rotate, stretch, mirror or shorten the perturbation vectors of point 0 on the surface. Then, the vectors for one triangle. componentsof D are given by A = B,A- B$, This way, he defineda bump-map function as a displace- ment but uses its derivatives at the point (u,v) to calculateD. D can be better expressedby 3.2. Hardware reflected ray Basedon the M*L model [IO], with homogeneous D = B,(N x 0,) - B,(N x 0,) (2) eye vector B = E and surfacenormal vector x = $ _ the homogeneous reflected ray in ScreenSpace where N/IN] is written as N. coordinatesleads to

3. SIMPLIFICATIONS 7l = [r,rYrz] 3.1. Hardware bump-mapping Hardware bump-mapping needs to be much = 2N(NE) - (NN)E simplerthan the classicalapproach [Equation (2)]. Instead of constructing a perturbation vector out of local derivatives which have to be defined, we already store it in the bump-map. Since we inter- (6) polate the normal over one triangle, we already have the tangent planeof the surface.Given the perturbation vector Pi = [AuAvlT in WC, we might construct = ~[r,r,r,] = 2--$N(NE) - -!-(NN)Ew3 the resulting normal vector as N= N+ P. To guarantee a perturbation in the direction O,, 0, of the tangent plane at point i, we have to align the = $(2N(NE) - (NN)E) = -$R texture coordinate systemu, v, w to the tangent plane system0,, 0,, N. This transformation is done by an Sincecalculating R in ScreenSpaceleads to higher alignment matrix A. Then the resulting normal precision multipliers and adders (extra bits for vector becomes N’= N+ A.P. Since we have a homogeneouscomponent), we convert E and N to homogeneousinterpolated normal vector, the equa- WC to reducethe overall gate-costsfor the Reflected tion expands to Ray Unit. Hardware-based Phong shading and bump mapping 145 3.3. Hardware Phong 3.3.2. Using DirectPhong@ (VISA+). The com- There are two ways to calculate the color mon problem for designersof hardware shadersis contribution of each lightsource. the exponentiation in the Phong formula. Gouraud 3.3.1. Using reflection maps (VISA). N and R are Shaderscalculate the intensitieson polygon-vertices used as an index into the diffuse and specular in software and interpolate these RGB values to reflection map respectively. producepixelvalues. The flaw in this approachis that Diffuse: no real, focusedhighlights can be achieved if the highlight lies within a polygon. This reduces the [wynz] + max { Inxll~yllnrl}, &+wynz) realismof the renderedscene. Attempts to implement Phong shading all use table lookups for the MapAddress = [cl,a,,] = exponentiation. This resultsin intolerable hardware sizes if different exponent values of must be X - Faces 6: supported. The VISA system implements Phong [ 1 Shading using reflectance maps. This is a fast solution to provide real-time Phong Shading with real highlights. The problem with this implementation is that it can only modelone exponentper scene. Currently, the VISA+ rendereris being designedat the GMD-FIRST that supports DirectPhong. The VISA+ system is based upon the concept that Note that the homogeneousperturbed normal is Phong’sexponentiation-function is purely empirical. N’/w (seeEquation (3)) which in the caseof X-Faces It implementsan atternative function to simulatethe leadsto shapeof Phong’sfunction and to provide real-time nynz Phong Shading for a broad range of specular -- exponents,defined by the OpenGL@ specifications. X-Faceslww= (7) It is beyond the scopeof this paper to describethe n,-- nx 1ww 1 VISA+ systemin more detail. Future publications will focus on this subject. As we can see,the divisionby w iscanceled out. This meansthat no conversionfrom and to homogeneous 3.4. Hardware environment mapping coordinatesis necessarywhen indexing the maps. Metallic or mirror-like objects can easily be Specular: simulatedusing the reflectancecube approach.Since The specularindex is calculatedanalogously to the the reflected view-vector must be calculated for diffuse index using R. modeling specular highlights, Phong Shading can easily be extendedto environment mapping. Closed X - Faces l&G room surroundingscan be stored in the cube maps. I 1 Each cube side contains the image of the specified For examplethe a, index expandsto direction (up, down, left, right, front, back). The reflectedview-vector will then beused for addressing

eY -n< + n: - nt + 2 . ny . (e, . n, + e, . n,) thesemaps, corresponding to the addressgeneration ( > for the reflection maps(Section 3.3). W: a, = In the example shown in Fig. 1, the surface e, . -nz - n: + nz + 2 . nz . (e,y . n, + e, . ny) ( > (perfect mirror) reflects the sky (top side) of the environment. Figure 2 illustrates the principal of W: cubeenvironment mapping. Even the slightestchange of the reflectancevector -nl+ns-nl + 2 . n, . (e, . n, + e, . n,) eY . ( ) a., = ez . -nf - n; + nt -t2~nz~(ex~nx+ey~ny) ( > Again we can use the index division for the transformation from ScreenSpacecoordinates to WC. As mentionedin Section3.2, N and E are already converted to WC, therefore we only need a divider without the additional precision bits for the homogeneouscomponent. To avoid distortion, an arcustangens table is used to correct the reflectancemap address. Fig. 1. Effect on interpolationtechniques using cube [axay]’ = [tan-’ (a,)tan-’ (ay)] environmentmapping. 146 K. Bennebroek, I. Ernst, H. Russ&r and 0. Wittig single patch

perfect mirror tesselated shapes Fig. 2. Principle of environment mapping. Fig. 4. Simulating curvature using edge normals.

3.6. Simulatedcurvature at the polygon edges produces visible artifacts, To reducethe complexity of curved objects course Therefore it is strongly recommended to use homo- triangulation can be used. A typical example for geneous interpolation for eye and normal vectors simulated curvature using different directions for (see image on the right in Fig. l), to get undistorted, edgenormals are tessellatedhemispheres. high quality results. The image on the left in Fig. 1 is In Fig. 4, a projection of a surfaceis shown with generated using linearly interpolated eye-vectors. As normalsperpendicular to this surface.Assuming that a result, the mirror image is shifted at the polygon this surface is a patch of a hemispherewith the edges. This results in missing perspective and shapesdrawn below, normalsare rotated about 45”, crooked illumination. 30” and lo”, respectively. However, to simulate a hemisphereby a cube’s 3.5. Hardware environment bump-mapping shapeis only usefulif it is very small. For example,it A powerful new feature in our hardware archi- might be useful for a level of detail representation, tecture is environment bump-mapping. Now the required for far away objects. modeler can incorporate structured mirrors into his Sincewe are interpolating normal vectors lineariy or her virtual environments. To accomplish this task, over the triangles, the effects discussedin the bump-mapping and cube enviromnent mapping is previoussections will also occur. combined. Fortunately, the variation of vertex normal vectors The perturbed reflection vector is calculated on the over a triangle are not as big as on eye vectors. In basis of the perturbed normal vector. This normal is our investigation on curvature we used large planes generated exactly in the same manner as if only with at least 800x800 drawn pixels, Therefore bump-mapping was used. shadingresults are extremely sensitiveto interpola- Figure 3 demonstrates the capabilities of this new tion artifacts. To ensurethat even small artifacts can graphics feature. For the images, we computed a be recognizedwe usedcube environment mapping bump-map with a small depth structure on our perfect with a grid texture. mirror surface. As a result, the sky is reflected perfectly with the bump structures taken into account. The image on the left hand shows linearly interpolated eye-vectors and therefore the same artifacts as in the case of pure cube environment mapping. Homoge- neously interpolated eye-vectors lead to artifact-free results (image on the right in Fig. 3).

Fig. 3. Linear(left) versushomogeneous (right) interpola- Fig. 5. Normal and eye-vectorsinterpolated homoge- tion for bump-environmentmapping. neously. Hardware-based Phong shading and bump mapping In the table below, images with 5” steps of normal vector angles are shown. Nearly invisible artifacts are in the range of 0’IO”, which leads to over 18x18 patches per hemisphere (see marked region in the 10” case). Acceptable results for medium sized polygons lie in the range of 10”30” (18x18-6x6). Angles greater than 30” are only useful for very small objects since they reveal severe artifacts (see marked regions in images). In Fig. 5, environment mapping with homogeneously interpolated normal and eye-vectors is shown. It can be Seenthat there are no distortions on the polygon edges and the simulated curvature (45” normal vector angle) is expressed in the bowed grid of the texture (Fig. 6).

4. INTERPOLATION Fig. 6. Simulated curvature. Since the variation of the eye vector over the surface is immense compared to the normal vector, we generated Fig. 7 to show the severe change in eye- directions. Figure 8 gives an impression of the architecture. Our investigations on interpolation techniques for The shaded boxes show the functional complexity of all critical parameters of our renderer design lead us the VISA + rasterizer engine, the BEAT (= Bump to the results depicted in Table 1. Environment And Texture)modules implement the functions necessary for memory relevant graphic operations with high bandwidth requirements. (The 5. PROPOSED ARCHlTECTURE BEAT module will be described in a further report.) The VISA + rendering engine scan-converts trian- We will focus on the functional units of the VISA+ gular datasets in a uniform manner [S]. Given a rasterizer. triangle vertex, the attached slope increments and The bump engine perturbs the pixel normal by the their attribute parameters (normal, eye, fog, etc.), the corresponding bump-map entries AU and Av. In interpolators first compute the edge parameters per order to align the bump-map vectors perpendicu- scanline and subsequently the interpolated attributes larly to the surface normal, a transformation within a scanline. As the rasterization unit inter- process must be executed by this functional unit. polates main shading parameters in homogeneous This is done by multiplying the bump-map entries space a division by w for u,v-texture and bump by matrix A. coordinates as well as for main vector entities The reflectance cubes are addressed for the (normal-, eye-, etc. vectors), is mandatory for screen determination of the diffuse component by the space conversion. The division of w for the relevant normal N and for the determination of specular parameters per pixel implies a screen-projective component by the reflected ray R. The vector’s major mapping. In contrast to bi-linear mapping techni- axis determines the corresponding reflectance cube ques, the above mentioned perspective-correct ren- face. Indexing of this map is done by dividing the dering generates artifact-free perspective scenes, no vector’s minor-axis values by the major-axis value. swimming textures, environments and bump-envir- As one can see, there is no special unit left for onments in an animation. environment bump-mapping. The VISA + engine 148 K. Bennebroek, I. Ernst, H. Rtisseler and 0. Wittig Table 1. Interpdation technique for various parameters Parameter Interpolation technique Normal-vector Homogeneous Recommended Eye-vector Homogeneous Strictly Texture Homogeneous Strictly Vertex color Linear Not critical 2 value Homogeneous Recommended Misc. (fog etc.) Linear Not critical

will work in this operation mode by storing the Due to the simplegeometry of the room walls and environment of an object in the reflectancecubes. the ground, Gouraud interpolation of the Vertex The object’s structure is stored as bump-texture in colors lead to the typical artifacts. In the Direct- the bump-module,the object’stexture in the texture Phongexample there is no visible artifact even if the module. The color blending unit combinesall these complexity of the walls is reducedto two triangles. different color sourcesto the blendedcolor of a pixel. Figure 12 showsthe affect of completely vanished For eachpass up to four spot lights or up to eight spotlights.The reasonfor this is, that the curtain is local lights can be generated according to the constructed out of triangles spanning the whole OpenGL illumination equation. curtain height. Thereforeexact illuminatedpixels can only be computedoutside the spotlight cutoff. 6. DIRECTPHONG@+ Calculating the full OpenGL@ lighting equation 7. CONCLUSIONS AND FUTURE WORK (Phong reflection model) on a per pixel basisgives The division by w of the relevant parameters great advantages over the per vertex calculation implies a screen-projectivemapping. In contrast to method usedfor Gouraud shading. bilinear mapping techniques, perspective-correct Figure 9-11, respectively, show: rendering generates artifact-free perspective scenes and prevents swimming textures, environments and l focusedhighlights bump-environmentsfrom occurring in an animation, l spot lights Currently, the VISA + architecture, supporting l light sourceattenuation (> 0, < 0) real-time Phong Shading with realistic texture-, bump- and environment-bump-mapping,is being + DirectPhong is trademark of GMD FIRST Berlin. designed at the GMD-FIRST. The rasterization unit

odor XY = Fig. 8. Overall architecture. Hardware-based Phong shading and bump mapping 149

Fig. 9. Phong: attenuated spot (left); brightened spot (right).

Fig. 10. Gouraud: attenuated spot (left); brightened spot (right).

Fig. 12. Spotlights: Phong (top); Gouraud (bottom).

2. Deering, M., Winner, S., Schediwy, B., Duffy, C. and Hunt, N., The triangle processor and normal vector shader: a VLSI system for high performance graphics. Computer Graphics, 1988, 22(4), 21-30. 3. Clausen, U., Reducing the Phong shading method. In Proceedings of Eurogruphics’89, 1989, pp. 333344. Fig. Il. Complexity of sample scene: wall, eight triangles; 4. Kuijk, A. M. and Blake, E. H., Faster Phong shading sphere, 800 triangles. via angular interpolation. Computer Gruphics Forum, 1989,8, 315-324. 5. Jackel, D. and Riisseler, H., A real time rendering system with normal vector shading. In Proceedings of the 9th EuroGruphics Workshop on Graphics Hurdware, will be fabricated in 0.35 micron technology. The Oslo, Norway, 1994. datapath is implementedin a full custom design 6. Voorhies, D. and Foran, J., Reflection vector shading style. We believe that full custom datapath design hardware. Computer Graphics (Proceedings SIG- style will be a must for high complexity designs GRAPH ‘94, 1994, 28, 163-166. minimizing the power consumption[l 11.The design I. Phong, B. T., Illumination for computer generated pictures. Communicutions of the ACM, 1975, 18(6), hasa complexity of circa onemillion gates(spot light 311-317. channelsinclusive). By the end of 1996,we expect a 8. Blinn, J. F., Simulation of wrinkled surfaces . Computer working prototype. Using this rendering-system,we Graphics (Proceedings SIGGRAPH ‘78), 1978,12,28& are able to generate animated scenesthat by far 292. 9. Ernst, I., Jackbl, D., Riisseler, H. and Wittig, O., surpassthe realism of conventional Gouraud-sha- Hardware supported bump mapping: a step towards ders. higher quality real-time rendering. In Proceedings of fOth EuroGraphics Workshop on Graphics Hurdwure, AcknowledgementsWe are grateful to S. Budianto, K. EuroGraphics, 1975,pp. 63-70. Tehrani and our former colleague T. Le Vin for their IO. Fischer, F. and Wor, A., R.E. versus N.H. specular suggestions and contributions to this work. highlights. In Graphic Gems, ed. P. S. Heckbert. Academic Press, Boston, 1994, pp. 388-400. Il. Smit, J. and Bosma, M., On the energy complexity of REFERENCES algorithms realized in CMOS, a graphics example. In 1. Bishop, G. and Weimar, D. M., Fast Phong shading. EuroGraphics Workshop on Gruphics Hardwure, Poi- Computer Graphics, 1986, 20, 255-262. tiers, France, 1996.