The Graphics Pipeline and OpenGL II: Lighting and Shading, Fragment Processing Gordon Wetzstein Stanford University EE 267 Virtual Reality Lecture 3 stanford.edu/class/ee267/ Announcements • Most likely: everyone on the wait list will get in. We may even have space for a few more students, so email us right away if you’re auditing and would like to take the class! • questions for HW1? post on piazza and zoom office hours! • WIM workshop 1: this Friday 2-3 pm, zoom à if you are a WIM student, you must attend! • WIM HW1 going out this Friday Lecture Overview • rasterization • the rendering equation, BRDFs • lighting: computer interaction between vertex/fragment and lights • Phong lighting • shading: how to assign color (i.e. based on lighting) to each fragment • Flat, Gouraud, Phong shading • vertex and fragment shaders • texture mapping Review of Vertex/Normal Transforms vertex/normal transforms https://www.ntu.edu.sg/home/ehchua/programming/opengl/CG_BasicsTheory.html Rasterization Rasterization Purpose: 1. determine which fragments are inside the triangles 2. interpolate vertex attributes (e.g. color) to all fragments https://www.ntu.edu.sg/home/ehchua/programming/opengl/CG_BasicsTheory.html Rasterization / Scanline Interpolation y • grid of 6x6 fragments 5 4 3 2 1 0 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y • grid of 6x6 fragments 5 • 2D vertex positions after transformations (x1, y1) 4 3 2 1 (x2, y2) (x3, y3) 0 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y • grid of 6x6 fragments 5 • 2D vertex positions after transformations (x1, y1) 4 + edges = triangle 3 2 1 (x2, y2) (x3, y3) 0 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y • grid of 6x6 fragments 5 • 2D vertex positions after transformations A1 4 + edges = triangle 3 • each vertex has 1 or more attributes A, 2 such as R/G/B color, depth, … 1 • user can assign arbitrary attributes, e.g. A2 A3 0 surface normals 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y • scanline moving top to bottom 5 A1 4 3 2 1 A2 A3 0 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y • scanline moving top to bottom 5 A1 4 3 2 1 A2 A3 0 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y • scanline moving top to bottom 5 • determine which fragments are inside the A1 4 triangle 3 2 1 A2 A3 0 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y • scanline moving top to bottom 5 • determine which fragments are inside the A1 4 triangle A(l) A(r 3 ) • interpolate attribute along edges in y 2 (l) (l) (l) ⎛ y − y2 ⎞ ⎛ y1 − y ⎞ A = A1 + A2 1 ⎜ ⎟ ⎜ ⎟ ⎝ y1 − y2 ⎠ ⎝ y1 − y2 ⎠ A2 A3 0 (r) (r) 0 1 2 3 4 5 x (r) ⎛ y − y3 ⎞ ⎛ y1 − y ⎞ A = ⎜ ⎟ A1 + ⎜ ⎟ A3 ⎝ y1 − y3 ⎠ ⎝ y1 − y3 ⎠ Rasterization / Scanline Interpolation y • scanline moving top to bottom 5 • determine which fragments are inside the A1 4 triangle A(l) A A(r) 3 • interpolate attribute along edges in y • then interpolate along x 2 1 A2 A3 (l) (r) ⎛ x − x ⎞ (r) ⎛ x − x ⎞ (l) 0 A A A = ⎜ (r) (l) ⎟ + ⎜ (r) (l) ⎟ 0 1 2 3 4 5 x ⎝ x − x ⎠ ⎝ x − x ⎠ Rasterization / Scanline Interpolation y repeat: 5 • interpolate attribute along edges in y A1 4 • then interpolate along x 3 A(l) A(r) 2 1 A2 A3 0 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y repeat: 5 • interpolate attribute along edges in y A1 4 • then interpolate along x 3 2 A(l) A(r) 1 A2 A3 0 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y 5 A1 4 3 2 1 A2 A3 0 0 1 2 3 4 5 x Rasterization / Scanline Interpolation y output: set of fragments inside triangle(s) 5 with interpolated attributes for each of 4 these fragments 3 2 1 0 0 1 2 3 4 5 x Lighting & Shading (how to determine color and what attributes to interpolate) The Rendering Equation • direct (local) illumination: light source à surface à eye • indirect (global) illumination: light source à surfaceà … à surface à eye Kajija “The Rendering Equation”, SIGGRAPH 1986 The Rendering Equation • direct (local) illumination: light source à surface à eye • indirect (global) illumination: light source à surfaceà … à surface à eye radiance towards viewer emitted radiance BRDF incident radiance from some direction Kajija “The Rendering Equation”, SIGGRAPH 1986 The Rendering Equation • direct (local) illumination: light source à surface à eye • indirect (global) illumination: light source à surfaceà … à surface à eye 3D location radiance towards viewer emitted radiance BRDF incident radiance from some direction Kajija “The Rendering Equation”, SIGGRAPH 1986 The Rendering Equation • direct (local) illumination: light source à surface à eye • indirect (global) illumination: light source à surfaceà … à surface à eye Direction towards viewer radiance towards viewer emitted radiance BRDF incident radiance from some direction Kajija “The Rendering Equation”, SIGGRAPH 1986 The Rendering Equation • direct (local) illumination: light source à surface à eye • indirect (global) illumination: light source à surfaceà … à surface à eye wavelength radiance towards viewer emitted radiance BRDF incident radiance from some direction Kajija “The Rendering Equation”, SIGGRAPH 1986 The Rendering Equation • direct (local) illumination: light source à surface à eye • indirect (global) illumination: light source à surfaceà … à surface à eye time radiance towards viewer emitted radiance BRDF incident radiance from some direction Kajija “The Rendering Equation”, SIGGRAPH 1986 The Rendering Equation • drop time, wavelength (RGB) & global illumination to make it simple • direct (local) illumination: light source à surface à eye • indirect (global) illumination: light source à surfaceà … à surface à eye Kajija “The Rendering Equation”, SIGGRAPH 1986 The Rendering Equation • drop time, wavelength (RGB), emission & global illumination to make it simple • direct (local) illumination: num _lights light source à surface à eye L0 (x,ω 0 ) = ∑ fr (x,ω k ,ω o )Li (x,ω k )(ω k ⋅n) k=1 • indirect (global) illumination: light source à surfaceà … à surface à eye Kajija “The Rendering Equation”, SIGGRAPH 1986 The Rendering Equation • drop time, wavelength (RGB), emission & global illumination to make it simple num _lights L0 (x,ω 0 ) = ∑ fr (x,ω k ,ω o )Li (x,ω k )(ω k ⋅n) k=1 Bidirectional Reflectance Distribution Function (BRDF) • many different BRDF models exist: analytic, data BRDF.en.html / driven (i.e. captured) GlobalIllumination /lecture/cg/ escience.anu.edu.au http:// Ngan et al. 2004 Bidirectional Reflectance Distribution Function (BRDF) • can approximate BRDF with a few simple BRDF.en.html / components GlobalIllumination /lecture/cg/ normal specular incident light component direction escience.anu.edu.au http:// diffuse component surface Phong Lighting • emissive part can be added if desired • calculate separately for each color channel: RGB Phong Lighting CG_BasicsTheory.html • simple model for direct lighting / opengl • ambient, diffuse, and specular parts /programming/ ehchua /home/ • requires: • material color mRGB (for each of www.ntu.edu.sg normalized vector pointing towards light source ambient, diffuse, specular) L https:// N normalized surface normal • light color lRGB (for each of ambient, normalized vector pointing towards viewer diffuse, specular) V R = 2(N i L)N − L normalized reflection on surface normal Phong Lighting: Ambient • independent of light/surface position, viewer, normal • basically adds some background color ambient ambient m{R,G,B} ⋅l{R,G,B} Phong Lighting: Diffuse CG_BasicsTheory.html • needs normal and light source direction / opengl • adds intensity cos-falloff with incident angle /programming/ ehchua /home/ www.ntu.edu.sg diffuse diffuse https:// m{R,G,B} ⋅l{R,G,B} ⋅max(L i N,0) dot product Phong Lighting: Specular CG_BasicsTheory.html • needs normal, light & viewer direction / opengl • models reflections = specular highlights /programming/ • shininess – exponent, larger for smaller ehchua /home/ highlights (more mirror-like surfaces) www.ntu.edu.sg https:// mspecular ⋅l specular ⋅max(R iV,0)shininess {R,G,B} {R,G,B} Phong Lighting: Attenuation CG_BasicsTheory.html • models the intensity falloff of light w.r.t. / opengl distance • The greater the distance, the lower the /programming/ intensity d ehchua /home/ 1 www.ntu.edu.sg 2 https:// kc + kld + kqd constant linear quadratic attenuation Phong Lighting: Putting it all Together • this is a simple, but efficient lighting model • has been used by OpenGL for ~25 years • absolutely NOT sufficient to generate photo-realistic renderings (take a computer graphics course for that) num _lights 1 color = mambient ⋅l ambient + mdiffuse ⋅l diffuse ⋅max(L i N,0) + mspecular ⋅l specular ⋅max(R iV,0)shininess {R,G,B} {R,G,B} {R,G,B} ∑ k + k d + k d 2 ( {R,G,B} i,{R,G,B} i {R,G,B} i{R,G,B} i ) i=1 c l i q i ambient attenuation diffuse specular Lighting Calculations • all lighting calculations happen in camera/view space! • transform vertices and normals into camera/view space • calculate lighting, i.e. per color (i.e., given material properties, light source color & position, vertex position, normal direction, viewer position) Lighting v Shading • lighting: interaction between light and surface (e.g. using Phong lighting model; think about this as “what formula is being used to calculate intensity/color”) • shading: how to compute color of each fragment (e.g. what attributes to interpolate and where to do the lighting calculation) 1. Flat shading 2. Gouraud shading (per-vertex lighting) 3. Phong shading (per-fragment lighting) - different from Phong lighting courtesy: Intergraph Computer Systems Flat Shading • compute color only once per triangle (i.e. with Phong lighting) • pro: usually fast to compute; con: creates a flat, unrealistic appearance • we won’t use it Gouraud or Per-vertex Shading • compute color once per vertex (i.e.