Real-time Shading: Sampling Procedural Shaders Wolfgang Heidrich The University of British Columbia Abstract the illumination is fixed, but the camera is free to move around in the scene. In this case, the shader needs to In interactive or real-time applications, naturally the be evaluated for many different reference viewing posi- complexity of tasks that can be performed on the fly tions, and during realtime rendering the texture for any is limited. For that reason it is likely that even with the given viewing direction can be interpolated from the current rate of development in graphics hardware, the reference images. This four-dimensional data structure more complex shaders will not be feasible in this kind (2 dimensions for u and v, and 2 dimensions for the of application for some time to come. camera position) is called a light field, and is briefly One solution to this problem seems to be a precom- described in Section 4. putation approach, where the procedural shader is eval- If we want to go one step further, and keep the illumi- uated (sampled), and the resulting values are stored in nation flexible as well, we end up with a even higher di- texture maps, which can then be applied in interactive mensional data structure. There are several ways to do rendering. A closer look, however, reveals several tech- this, but one of the more promising is probably the use nical difficulties with this approach. These will be dis- of a space-variant BRDF, i.e. a reflection model whose cussed in this section, and hints towards possible solu- parameters can change over a surface. This yields an tions will be given. approach with a six-dimensional data structure that will be outlined in Section 5. No matter which of these approaches is to be taken, 1 Introduction and Problem State- there are some issues that have to be resolved for all ment of them. One of them is the choice of an appropriate resolution for the sampling process. The best resolu- In order to come up with an approach for sampling pro- tion depends on many different factors, some of which cedural shaders, we first have to determine which as- depend on the system (i.e. the amount of memory avail- pects of the shading system we would like to alter in able, or the range of viewing distances under which the the interactive application. shaded object will be seen), and some of which depend For example, we can evaluate the shader for a num- on the shader (i.e. the amount of detail generated by the ber of surface locations with fixed illumination (all light shader). source positions and parameters are fixed), and a fixed In the case of a 2D texture with fixed camera and camera position. This is the mode in which a normal lighting, a sample resolution can still be chosen rela- procedural shading system would evaluate the shader tively easily, for example, by letting the user make a for a surface in any given frame. If the shader is ap- decision. With complex view-dependent effects this is plied to a parametric surface F (u; v), then we can eval- much harder because it is hard to determine appropri- uate the shader at a number of discrete points (u; v), ate resolutions for sampling specular highlights whose and store the resulting color values in a texture map. sharpness may vary over the surface. An automatic In an interactive application, however, this particular method for estimating the resolution would be highly example is of limited use since both the viewer and the desirable. illumination is fixed. As a result the texture can only Another problem is the sheer number of samples that be used for exactly one frame, unless the material is we may have to acquire. For example, to sample a completely diffuse. In a more interesting scenario, only shader as a space variant BRDF with a resolution of 9 – 1 256x256 for the surface parameters u and v, as well as AA also maintains dependencies between the sources 322 samples for both the light direction and the viewing of error, and thus manages to compute significantly direction requires over 68 billion samples, which is un- tighter error bounds. Detailed comparisons between in- feasible both in terms of memory consumption and the terval arithmetic and affine arithmetic can be found in time required to acquire these samples. On the other [4], [5], and [6]. hand, it is to be expected that the shader function is rel- Affine arithmetic operates on quantities known as atively smooth, with the high-frequency detail localized affine forms, given as polynomials of degree one in a in certain combinations of viewing and light directions set of noise symbols i. (specular highlights, for example). Thus, a hierarchical x^ = x + x + x + · · · + xnn sampling scheme is desirable which allows us to refine 0 1 1 2 2 the sampling in areas that are more complex without The coefficients xi are known real values, while the val- having to do a high-density sampling in all areas. At the ues of the noise symbols are unknown, but limited to same time the hierarchical method should make sure we the interval U := [−1; 1]. Thus, if all noise symbols do not miss out on any important features. Such an ap- can independently vary between −1 and 1, the range of proach is described in the next section. possible values of an affine form x^ is n [x^] = [x0 − ξ; x0 + ξ]; ξ = X jxij: 2 Area Sampling of Procedural i=1 Shaders Computing with affine forms is a matter of replac- ing each elementary operation f(x) on real numbers ∗ In this section we introduce the concept of area sam- with an analogous operation f (1; : : : ; n) := f(x^) pling a procedural shader using a a certain kind of arith- on affine forms. metic that replaces the standard floating point arith- If f is itself an affine function of its arguments, we metic. This affine arithmetic allows us to evaluate a can apply normal polynomial arithmetic to find the cor- shader over a whole area, yielding an upper and a lower responding operation f ∗. For example we get bound for all the values that the shader takes on over x^ + y^ = (x + y ) + (x + y ) + · · · + (x + y ) this area. This bound can then be used hierarchically to 0 0 1 1 1 n n n refine the sampling in areas in which the upper and the x^ + α = (x0 + α) + x11 + · · · + xnn lower bound are far apart (i.e. areas with a lot of detail). αx^ = αx0 + αx11 + · · · + αxnn The full details of the method can be found in [10]. for affine forms x;^ y^ and real values α. We will discuss the general approach in terms of sampling a 2D texture by evaluating a shader with a fixed camera position and illumination. The same 3 Non-Affine Operations methods can however be applied to adaptively adjust If f is not an affine operation, the corresponding func- the sampling rates for camera and light position. ∗ tion f (1; : : : ; n) cannot be exactly represented as a linear polynomial in the i. In this case it is necessary a 2.1 Affine Arithmetic to find an affine function f (1; : : : ; n) = z0 + z11 + · · ·+z f ∗( ; : : : ; ) Affine arithmetic (AA), first introduced in [4], is an n n approximating 1 n as well as pos- Un extension of interval arithmetic [16]. It has been suc- sible over . An additional new noise symbol k has cessfully applied to several problems for which interval to be added to represent the error introduced by this ap- arithmetic had been used before [17, 20, 21]. This in- proximation. This yields the following affine form for z = f(x) cludes reliable intersection tests of rays with implicit the operation : a surfaces, and recursive enumerations of implicit sur- z^ = f (1; : : : ; n) = z0 + z11 + · · · + znn + zkk; faces in quad-tree like structures [5, 6]. with k 62 f1; : : : ; ng. The coefficient z of the new Like interval arithmetic, AA can be used to manip- k noise symbol has to be an upper bound for the error ulate imprecise values, and to evaluate functions over introduced by the approximation of f ∗ with f a: intervals. It is also possible to keep track of truncation ∗ a and round-off errors. In contrast to interval arithmetic, zk ≥ maxfjf (1; : : : ; n)−f (1; : : : ; n)j : i 2 Ug: 9 – 2 For example it turns out (see [4] for details) that a Shading languages usually introduce a set of spe- good approximation for the multiplication of two affine cific data types and functions exceeding the functional- forms x^ and y^ is ity of general purpose languages and libraries. Most of these additional functions can easily be approximated z^ = x0y0+(x0y1+y0x1)1+· · ·+(x0yn+y0xn)n+uvk; by affine forms using techniques similar to the ones out- lined in the previous section. Examples for this kind of n n with u = Pi=1 jxij and v = Pi=1 jyij. In general, the domain specific functions are continuous and discon- best approximation f a of f ∗ minimizes the Chebyshev tinuous transitions between two values, like step func- error between the two functions. tions, clamping of a value to an interval, or smooth Her- The generation of affine approximations for most of mite interpolation between two values. the functions in the standard math library is relatively The more complicated features include splines, straightforward. For a univariate function f(x), the iso- pseudo-random noise, and derivatives of expressions.