Depth and Transparency Computer Graphics CMU 15-462/15-662 Today: Wrap up the rasterization pipeline! Remember our goal: • Start with INPUTS (triangles) –possibly w/ other data (e.g., colors or texture coordinates) • Apply a series of transformations: STAGES of pipeline • Produce OUTPUT (fnal image) INPUT RASTERIZATION OUTPUT (TRIANGLES) PIPELINE (BITMAP IMAGE) VERTICES A: ( 1, 1, 1 ) E: ( 1, 1,-1 ) B: (-1, 1, 1 ) F: (-1, 1,-1 ) C: ( 1,-1, 1 ) G: ( 1,-1,-1 ) D: (-1,-1, 1 ) H: (-1,-1,-1 ) TRIANGLES EHF, GFH, FGB, CBG, GHC, DCH, ABD, CDB, HED, ADE, EFA, BAF CMU 15-462/662 What we know how to do so far… (w, h) y z x (0, 0) position objects in the world project objects onto the screen sample triangle coverage (3D transformations) (perspective projection) (rasterization) ??? put samples into frame buffer sample texture maps interpolate vertex attributes (depth & alpha) (fltering, mipmapping) (barycentric coodinates) CMU 15-462/662 Occlusion CMU 15-462/662 Occlusion: which triangle is visible at each covered sample point? Opaque Triangles 50% transparent triangles CMU 15-462/662 Sampling Depth Assume we have a triangle given by: – the projected 2D coordinates (xi,yi) of each vertex – the “depth” di of each vertex (i.e., distance from the viewer) screen Q: How do we compute the depth d at a given sample point (x,y)? A: Interpolate it using barycentric coordinates (just like any other attribute that varies linearly over the triangle) CMU 15-462/662 The depth-buffer (Z-buffer) For each coverage sample point, depth-buffer stores the depth of the closest triangle seen so far. Initial state of depth buffer before rendering any triangles (all samples store farthest distance) Grayscale value of sample point used to indicate distance Black = small distance White = large distance (“infnity”) CMU 15-462/662 Depth buffer example CMU 15-462/662 Example: rendering three opaque triangles CMU 15-462/662 Occlusion using the depth-buffer (Z-buffer) Processing yellow triangle: Grayscale value of sample point depth = 0.5 used to indicate distance White = large distance Black = small distance Red = sample passed depth test Color buffer contents Depth buffer contents CMU 15-462/662 Occlusion using the depth-buffer (Z-buffer) After processing yellow triangle: Grayscale value of sample point used to indicate distance White = large distance Black = small distance Red = sample passed depth test Color buffer contents Depth buffer contents CMU 15-462/662 Occlusion using the depth-buffer (Z-buffer) Processing blue triangle: Grayscale value of sample point depth = 0.75 used to indicate distance White = large distance Black = small distance Red = sample passed depth test Color buffer contents Depth buffer contents CMU 15-462/662 Occlusion using the depth-buffer (Z-buffer) After processing blue triangle: Grayscale value of sample point used to indicate distance White = large distance Black = small distance Red = sample passed depth test Color buffer contents Depth buffer contents CMU 15-462/662 Occlusion using the depth-buffer (Z-buffer) Processing red triangle: Grayscale value of sample point depth = 0.25 used to indicate distance White = large distance Black = small distance Red = sample passed depth test Color buffer contents Depth buffer contents CMU 15-462/662 Occlusion using the depth-buffer (Z-buffer) After processing red triangle: Grayscale value of sample point used to indicate distance White = large distance Black = small distance Red = sample passed depth test Color buffer contents Depth buffer contents CMU 15-462/662 Occlusion using the depth buffer bool pass_depth_test(d1, d2) { return d1 < d2; } depth_test(tri_d, tri_color, x, y) { if (pass_depth_test(tri_d, zbuffer[x][y]) { // triangle is closest object seen so far at this // sample point. Update depth and color buffers. zbuffer[x][y] = tri_d; // update zbuffer color[x][y] = tri_color; // update color buffer } } CMU 15-462/662 Does depth-buffer algorithm handle interpenetrating surfaces? Of course! Occlusion test is based on depth of triangles at a given sample point. The relative depth of triangles may be different at different sample points. Green triangle in front of yellow triangle Yellow triangle in front of green triangle CMU 15-462/662 Does depth-buffer algorithm handle interpenetrating surfaces? Of course! Occlusion test is based on depth of triangles at a given sample point. The relative depth of triangles may be different at different sample points. CMU 15-462/662 Does depth buffer work with super sampling? Of course! Occlusion test is per sample, not per pixel! This example: green triangle occludes yellow triangle CMU 15-462/662 Color buffer contents CMU 15-462/662 Color buffer contents (4 samples per pixel) CMU 15-462/662 Final resampled result Note anti-aliasing of edge due to fltering of green and yellow samples. CMU 15-462/662 Summary: occlusion using a depth buffer ▪ Store one depth value per coverage sample (not per pixel!) ▪ Constant space per sample - Implication: constant space for depth buffer ▪ Constant time occlusion test per covered sample - Read-modify write of depth buffer if “pass” depth test - Just a read if “fail” ▪ Not specifc to triangles: only requires that surface depth can be evaluated at a screen sample point But what about semi-transparent surfaces? CMU 15-462/662 Compositing CMU 15-462/662 Representing opacity as alpha Alpha describes the opacity of an object - Fully opaque surface: ! = 1 - 50% transparent surface: ! = 0.5 - Fully transparent surface: ! = 0 Red triangle with decreasing opacity ! = 1 ! = 0.75 ! = 0.5 ! = 0.25 ! =0 CMU 15-462/662 Alpha: additional channel of image (RGBA) ! of foreground object CMU 15-462/662 Over operator: Composite image B with opacity !B over image A with opacity !A A A over B != B over A B B “Over” is not commutative A B over A A over B Koala over NYC CMU 15-462/662 Fringing Poor treatment of color/alpha can yield dark “fringing”: foreground color foreground alpha background color fringing no fringing CMU 15-462/662 No fringing CMU 15-462/662 Fringing (…why does this happen?) CMU 15-462/662 Lecture 5 Math Lecture 5 Math d (x, y)=Ax + By + C wd (x, y)=Ax + By + C w T p0x p0y ,d0 T p0x p0y T ,d0 ⇥p1x p1y⇤ ,d1 T ⇥p1x p1y⇤T ,d1 ⇥p2x p2y⇤ ,d2 LectureT 5 Math ⇥p2x p2y⇤⇤ ,d2 C =⇥↵ B + (1⇤ ↵ )↵ A B − B A C = ↵BB +d (1 ↵B)↵AA Over operator:(−x, y)= non-premultipliedAx + By + C alpha ↵ = ↵ + (1 ↵ )↵ C B w − B A Composite image B with opacity !B over image A with opacity !A ↵C = ↵B + (1 ↵B)↵A A frst attempt: − T T A = Ar Ag pA0xb p0y ,d0 T A = ⇥A A A ⇤ r g⇥ b ⇤T A p1x T p1y ,d1 B B = ⇥Br Bg Bb ⇤ T T B = ⇥B B⇥p2Bx⇤ p2y⇤ ,d2 B over A r g b Appearance of semi- transparent A Composited⇥ color: ⇥ ⇤ ⇤ C = ↵ B + (1 ↵ )↵ A B − B A Appearance of What B lets through semi-transparent B ↵ = ↵ + (1 ↵ )↵ C B − B A CMU 15-462/662 Lecture 5 Math Lecture 5 Math d (x, y)=Ax + By + C wd (x, y)=LectureAx + By + 5C Math w T p0x p0y ,d0 T p d p T ,d ⇥p0x (px,0y y⇤)=,dAx0 + By + C 1xw 1y 1 ⇥ ⇤T pLecture1x p1y T ,d 5 Math1 ⇥p2x p2y⇤ ,d2 ⇥ ⇤T T ⇥p2x pp2y0⇤x ,dp0y2 ,d0 T C =⇥d↵BB +⇥ (1⇤ ↵B)↵⇤AA (x, y)=p1xAx− p+1yBy,d+ C1 COver=w↵ B +operator: (1 ↵ )↵ A premultiplied alpha B ⇥ B ⇤TA ⇥p2x− p2y⇤ ,d2 Composite↵C = ↵B + image (1 ↵ TBB with)↵A opacity B over image A with opacity A p0x⇥ p0−y ,d⇤ 0 ! ! Not premultiplied: ↵C Lecture= ↵B + (1 5↵ MathB)↵A C⇥ = ↵ B−⇤+T (1 ↵ )↵ A Non-premultipliedCp=1x ↵BpB1y + alpha: (1,dT 1↵B)↵AA A = Ar Ag Ab − ↵ = ↵ +T (1 ↵ )↵ ⇥p↵C =p↵B⇤+,d (1T ↵B)↵A dA = ⇥A2rx Ag2y Ab⇤ −2 (x, y)=Ax + By + C T A w A = Ar ATg Ab B B =⇥⇥Br Bg ⇤Bb ⇤ ⇥ T ⇤T BC == ⇥↵BBBB=B+⇥B (1rB ⇤B↵Bg )↵BAbA⇤ two multiplies, one add r g T −b B over A p0x p0y ,d0 (referring to vector ops on colors) Premultiplied: ⇥ ⇥ ⇤ ⇤ C0 = BT+ (1 ↵B)A Premultiplied⇥p1x 0 p1 alpha:y⇤ ,d1 B ↵C = ↵B + (1 ↵−B)↵A − T A0 = ↵AAr ↵TAAg ↵AAb ↵A ⇥p2x p2y⇤ ,d2 T B = ⇥↵ B ↵ B ↵ B ↵ ⇤ T B0 =⇥ ↵BBr ⇤↵BBg ↵BBb ↵B ⇥ ⇤ one multiply, one add C = ↵⇥ B + (1 ↵ )↵ A ⇤ B − B A Composite alpha: Notice premultiplied alpha composites alpha just like how it composites rgb. ↵C = ↵B + (1 ↵B)↵A − Non-premultiplied alpha composites alpha differently than rgb. CMU 15-462/662 A problem with non-premultiplied alpha ! Suppose we upsample an image w/ an alpha mask, then composite it onto a background ! How should we compute the interpolated color/alpha values? ! If we interpolate color and alpha separately, then blend using the non-premultiplied “over” operator, here’s what happens: original original color alpha upsampled upsampled color alpha Notice black “fringe” that occurs because we’re blending, e.g., 50% blue pixels using 50% alpha, rather than, say, 100% blue pixels with 50% alpha. composited onto yellow background CMU 15-462/662 Eliminating fringe w/ premultiplied “over” ! If we instead use the premultiplied “over” operation, we get the correct alpha: (1-alpha) background + = upsampled color (1-alpha)*background composite image w/ no fringe CMU 15-462/662 Eliminating fringe w/ premultiplied “over” ! If we instead use the premultiplied “over” operation, we get the correct alpha: composite image WITH fringe CMU 15-462/662 Similar problem with non-premultiplied alpha Consider pre-fltering (downsampling) a texture with an alpha matte Desired fltered result ! input color input ! fltered color fltered ! fltered result Downsampling non-premultiplied alpha composited over white image results in 50% opaque brown) 0.25 * ((0, 1, 0, 1) + (0, 1, 0, 1) + Result of fltering (0, 0, 0, 0) + (0, 0, 0, 0)) = (0, 0.5, 0, 0.5) premultiplied image CMU 15-462/662 LectureLecture 5 Math 5 Math Lecture 5 Math d d Lecture(x, y)=Ax 5+ MathBy + C (x,wd y)=Ax + By + C w (x, y)=Ax +