First Order Approximation for Texture Filtering Zhouchen Lin Lifeng Wang Liang Wan Microsoft Research, Asia Microsoft Research, Asia The Chinese University of Hong Kong [email protected] [email protected] [email protected] Abstract bic interpolation kernels, are spatially-invariant and inde- pendent of the mapping function. They are known to pro- Texture mapping is a core technique in 3D graphics and duce aliasing when filtering textures with high frequencies. is important for computer vision applications as well. The In contrast, spatially-variant filters are better at reduc- major challenge of texture mapping is high quality anti- ing the aliasing. Among them, MIPmap [14] is widely aliasing at a reasonably high speed. Previous approaches used because of its low computational cost. It creates tex- do not address this problem adequately, since they do not ture pyramids to manage texture level-of-detail (LOD) but precisely model the anisotropic and spatially variant nature uses square pixel approximation with poor weighting, hence in texture mapping. resulting in overly-blurred and poor off-angle quality tex- In this paper, we handle this problem using signal tures. Texture potential MIPmap [1] overcomes the problem processing and sampling theory. We first describe an an- of texture over-blurring by building a pyramid of potential alytic framework to deduce the ideal filter and the corre- maps, and finding the texture values by simple subtraction sponding weight distribution for texture filtering. Since the operations. ideal filter does not have a closed-form solution, we further Several hardware anisotropic filtering algorithms can propose a filter that is the first order, yet of high precision, produce better results than MIPmap, such as Textram [11], approximation of the ideal filter and makes a closed-form Feline [8], fast footprint MIPmap [6], and Talisman [12]. solution possible. This first order approximating (FOA) fil- Texram assumes a square pixel shape with a parallelo- ter has excellent anti-aliasing capability and a reasonably gram footprint, and computes with equal weights. Fe- high rendering speed. The comparison with some well- line uses several isotropic probes along a line to imple- known filters (box, cubic, EWA, and fast footprint MIPmap- ment an anisotropic filter with Gaussian weights. Fast foot- ping, etc.) also testifies that our filter does have better anti- print MIPmap also adopts the square pixel assumption with aliasing performance. quadrilateral footprint using MIPmap data. It computes the weights in the quadrilateral footprint with a Gaussian ker- nel. Talisman applies hardware acceleration, and assumes a 1. Introduction square pixel shape with quadrangle footprint. The weights are achieved from pre-computed MIPmap. Texture mapping is critical to all 3D graphics rendering Previous filters, however, typically treat texture mapping systems by making rendered objects more realistic. This in a rather ad hoc fashion. The elliptical weight average technology is also important for computer vision, especially (EWA) filter [4, 5, 2] is considered as a benchmark against analysis-by-synthesis techniques for extracting model para- which all texture mapping methods are compared. It uses meters. Examples include image-based triangulation [10] circular pixel shape (with elliptical footprint) and Gaussian and facial modeling fitting [7]. The major challenge of tex- weights to calculate mapped textures. The EWA filter could ture mapping is anti-aliasing. The ideal texture filter is im- be proven from the sampling theory to be close to, but still practical as it requires weighting among infinite number of not, the ideal filter for either reconstructing or prefiltering samples. Therefore, finding a practical texture filter with continuous signals. Thus the EWA filter still produces alias- high anti-aliasing capability and relatively high rendering ing effects when filtering textures with very high frequen- speed is very useful. cies and artifacts when up-sampling low frequency signals. Generally speaking, the process of texture mapping can Effort has been made to approximate the ideal filter [13, 3], be considered as resampling a source texture to create a but only partial approximations are obtained. destination texture according to a mapping function. Many In this paper, we start from basic signal processing and well known kernels, such as the box, the tent, and the cu- sampling theory to derive the ideal weight distribution for ~ ~ texture mapping. Next, we derive a first order approximat- f(k) f (l ) ing (FOA) filter which is a good approximation to the ideal filter. We also analyze this filter thoroughly and show that it has a closed-form solution. Finally we compare our pro- x ~ posed filter with typical texture mapping methods in 2D/3D x ~ ~ ~ bl scenarios. Reconstruct f c (x) ~ Sample fc(x) f c (x) Warp Prefilter 2. Theory of texture filtering ~ x x ~ x Texture mapping is a procedure that warps a source tex- ture f onto a surface through a transform T , and prefilters Figure 1. Steps involved in resampling. the warped texture to produce the anti-aliased destination Adapted from [15]. texture f~. It is actually a resampling process. Ideally, to compute the ideal filter associated with T , we identify the as the ideal weighting function. It determines the interpola- following steps [15]: tion weights from a source pixel f(k) to a destination pixel f~(~l). ² Reconstruct a continuous signal fc from a discrete sig- Unfortunately, equations (5) and (6) have no closed-form nal f using a sinc filter, i.e., solutions unless T is special. The numerical evaluation is X impractical, considering the intensive computational cost fc(x) = f(k)sinc(x ¡ k): (1) due to the infinite support of the sinc function. As a result, k approximations should be performed so that a closed-form solution is possible. In our approach, we take the first order ² Map the continuous signal fc to the desired continuous approximation of T to produce a closed-form solution. The ~ signal fc using the transform T , i.e., resultant filter is very close to the ideal one and produces excellent visual results compared to the previous ones. ~ fc(x~) = fc(T (x~)): (2) 3. First order approximating (FOA) filter ~ ² Filter fc using a sinc function so that it is band-limited, i.e., We use the first order Taylor series expansion to locally Z linearize the transform T in (6), which produces a local ~bl ~ ~ ~ ~ affine transform [4, 5, 8, 9]. Based on our analysis and fc (x~) = fc(t)sinc(x~ ¡ t)dt: (3) experimental results, this first order approximation is good enough. More importantly, this makes possible a closed- ~bl ~ ² Sample fc to produce the discrete output f, i.e., form solution of (6). The first order approximation of T (~t) is: f~(~l) = f~bl(~l): (4) c T (~t) = T (~l + (~t ¡ ~l)) ¼ T (~l) + J(~l)(~t ¡ ~l); These four steps are illustrated in Figure 1, where we use where J is the Jacobian matrix of T . Such an approximation the embellishment (~) to indicate those variables in the des- has a reasonably high accuracy. Even ~t is far from ~l, the tination domain. By plugging (1), (2), and (3) into (4) re- deviation from the exact value is still quite small, since the cursively, we have: modulation of sinc(~l ¡ ~t) in (6) decays quite fast. So we have: X Z ~ ~ ~ ~ ~ ~ ~ f(l) = f(k) sinc(T (t) ¡ k)sinc(l ¡ t)dt: ZW (l; k) k ¼ sinc(T (~l) + J(~l)(~t ¡ ~l) ¡ k)sinc(~l ¡ ~t)d~t We refer to Z ~ ~ ¡1 ~ ~ ~ Z = sinc(J(l)f[J(l)] [T (l) ¡ k] + (~t ¡ l)g)sinc(l ¡ ~t)d~t: h(x~; k) ´ sinc(T (~t) ¡ k)sinc(x~ ¡ ~t)d~t (5) Let ^t = ~l ¡ ~t. We can rewrite the interpolation kernel as Z as the ideal filter, and its discrete version: W (~l; k) = sinc(J(~l)f[J(~l)]¡1[T (~l) ¡ k] ¡ ^tg)sinc(^t)d^t Z ¯ ¯ W (~l; k) = sinc(T (~t) ¡ k)sinc(~l ¡ ~t)d~t (6) = [sinc(J(~l)x~) ­ sinc(x~)]¯ ; x~=[J(~l)]¡1[T (~l)¡k] where ­ is the convolution. Denote For a 1D signal (Figure 2), C~ is simply a scaled version of C, then h(x~) = sinc(J(~l)x~) ­ sinc(x~): (7) 1 ¡1 h(~x) = F [BOX ~ (!)](~x) It is the first order approximating filter. Then J(~l) minfJ(l);1g minfJ(~l); 1g W (~l; k) = h([J(~l)]¡1[T (~l) ¡ k]); (8) = sinc(minfJ(~l); 1gx~): J(~l) which is the first order approximating weight distribution. Therefore, To find the closed-form solution for (8), we apply the Fourier transform. It is well known that with the Fourier à ! minfJ(~l); 1g minfJ(~l); 1g transform, convolution becomes product and a sinc function W (~l; k) = sinc [T (~l) ¡ k] J(~l) J(~l) corresponds to a BOX function over a unit cube C. Let F 8 ~ ~ denote the Fourier transform, then < sinc(T (l)µ¡ k) ¶ if J(l) · 1 = 1 1 : : sinc (T (~l) ¡ k) otherwise F[sinc(J(~l)x~) ­ sinc(x~)](!) J(~l) J(~l) = F[sinc(J(~l)x~)](!) ¢ F[sinc(x~)](!) ~ ¡1 ~ ¡1 = j[J(l)] jBOXC([J(l)] !)BOXC(!) For the 2D case, C~ is a parallelogram. Figure 3 shows the ~ ¡1 ~ = jJ(l)j BOXC~ (!)BOXC(!) common part of C and C after clipping against each other. ~ ¡1 = jJ(l)j BOXC~\C(!); We find that there is a closed-form solution to the inverse Fourier transform of the clipped box function. where C~ = [J(~l)]C is the parallelogram transformed from C via the Jacobian J. Therefore, ¡1 ¡1 h(x~) = F [jJ(~l)j BOX~ (!)](x~) C\C x = ~ ¡1 ¡1 (9) = jJ(l)j F [BOXC~\C(!)](x~): x x x In the following, we present the closed-form solutions y in the 1D and the 2D cases, respectively.