Outline of Lecture

Real-Time High Quality Rendering §. Basics of hardware pipeline §. Reflection and Rendering equations CSE 274 [Fall 2015], Lecture 2 §. Typical Lighting, shading in hardware Graphics Hardware Pipeline, Reflection and §. Taxonomy of methods/papers Rendering Equations, Taxonomy of Methods

http://www.cs.ucsd.edu/~ravir §. Assignment: Sign up for paper presentations §. And send basic info: §. Name, e-mail, status (Senior, PhD etc.) Background in graphics/comments §. Will you be taking course grades or P/F Basic concepts/review only. If interested in more background/in depth detail, refer to handouts

Basic Hardware Pipeline Geometry or Vertex Pipeline

Application Geometry Rasterizer

CPU GPU Model, View Lighting Projection Clipping Screen Transform

These fixed function stages can be replaced by a general per-vertex calculation using vertex in modern programmable hardware

Create geometry, lights, Transform and lighting calcs. materials, textures, Apply per-vertex operations Textures, Cubemaps cubemaps, … as inputs Per-pixel (per-fragment) operations

Pixel or Fragment Pipeline GPU Programmable Shaders Programmable in Modern GPUs Programmable in (Vertex ) Modern GPUs Rasterization Texture Z-buffering Geometry (Fragment (scan conversion) Mapping Vertices Primitive Shader)

Scan Framebuffer Operations Fragment Conversion Operations (Rasterize) These fixed function stages can be replaced by a general per-fragment calculation using fragment shaders in modern programmable hardware Pixel Texture Images Operations Memory

Traditional Approach: Fixed function pipeline (state machine) New Development (2003-): Programmable pipeline

1 Outline of Lecture Reflection Equation

§. Basics of hardware pipeline §. Reflection and Rendering equations §. Typical Lighting, shading in hardware ω §. Taxonomy of methods/papers i ωr x §. Assignment: Sign up for paper presentations

L (x,ω ) = L (x,ω ) + L (x,ω )f (x,ω ,ω )(ω i n) r r e r i i i r i Basic concepts/review only. If interested in more Reflected Light Emission Incident BRDF Cosine of background/in depth detail, refer to handouts (Output Image) Light (from Incident angle light source)

Reflection Equation Reflection Equation

ω ω ω ω i r dωi i r x x Replace sum with integral Sum over all light sources Lx(,ω )=+ L (, xω ) Lx (,ωω )(,fx ωθ , )cos dω L (x,ω ) = L (x,ω ) + L (x,ω )f (x,ω ,ω )(ω i n) r r e r∫ i i ir i i r r e r ∑ i i i r i Ω Reflected Light Emission Incident BRDF Cosine of Reflected Light Emission Incident BRDF Cosine of (Output Image) Light (from Incident angle (Output Image) Light (from Incident angle light source) light source)

Rendering Equation Rendering Equation (Kajiya 86) Surfaces (interreflection) x′ dA

ω dωi i ωr x

L (x,ω ) = L (x,ω ) + L (x′,−ω )f (x,ω ,ω ) cosθ dω r r e r ∫ r i i r i i Ω Reflected Light Emission Reflected BRDF Cosine of (Output Image) Light Incident angle UNKNOWN KNOWN UNKNOWN KNOWN KNOWN

2 Rendering Equation as Integral Equation Linear Operator Equation

L (x,ω ) = L (x,ω ) + L (x′,−ω ) f (x,ω ,ω ) cosθ dω r r e r ∫ r i i r i i Ω lu()=+eu()lv () Ku (,)v dv Reflected Light Emission Reflected BRDF Cosine of ∫ (Output Image) Light Incident angle Kernel of equation UNKNOWN KNOWN UNKNOWN KNOWN KNOWN Light Transport Operator

Is a Fredholm Integral Equation of second kind [extensively studied numerically] with canonical form L = E + KL l(u) = e(u) + l(v) K(u,v)dv Can be discretized to a simple matrix equation ∫ [or system of simultaneous linear equations] Kernel of equation (L, E are vectors, K is the light transport matrix)

Ray Tracing and extensions Ray Tracing

– General class numerical Monte Carlo methods 2 3 – Approximate set of all paths of light in scene L = E + KE + K E + K E + ... L = E + KL Emission directly From light sources IL − KL = E Direct Illumination (I − K)L = E on surfaces −1 L = (I − K) E Global Illumination Binomial Theorem (One bounce indirect) L = (I + K + K 2 + K 3 + ...)E [Mirrors, Refraction] 2 3 (Two bounce indirect) L = E + KE + K E + K E + ... [Caustics etc]

Ray Tracing Outline of Lecture 2 3 L = E + KE + K E + K E + ... §. Basics of hardware pipeline §. Reflection and Rendering equations Emission directly §. Typical Lighting, shading in hardware From light sources §. Taxonomy of methods/papers Direct Illumination on surfaces §. Assignment: Sign up for paper presentations Global Illumination OpenGL Shading (One bounce indirect) [Mirrors, Refraction] Basic concepts/review only. If interested in more (Two bounce indirect) background/in depth detail, refer to handouts [Caustics etc]

3 OpenGL: Sum of Components Ambient term

Sum over all light sources L (x,ω ) = L (x,ω ) + L f (ω ,ω )(ω i n) r r e r ∑ i i r i Global constant (sometimes per light) added to everything Reflected Light Emission Incident BRDF Cosine of Incident angle (Output Image) Light (from §. In addition to other terms in reflection equation light source)

§. Ambient §. Fakes indirect illumination, broad area lights §. Emission §. Prevents completely black regions etc. §. Diffuse §. Hack, no physical basis §. Specular

Emissive Term (for Light Sources) Diffuse Term (Lambertian Reflection)

Sum over all light sources Sum over all light sources L (x,ω ) = L (x,ω ) + L f (ω ,ω )(ω i n) L (x,ω ) = L (x,ω ) + L f (ω i n) r r e r ∑ i i r i r r e r ∑ i d i Reflected Light Emission Incident BRDF Cosine of Reflected Light Emission Incident albedo Cosine of (Output Image) Light (from Incident angle (Output Image) Light (from Incident angle light source) light source)

BRDF constant (Lambertian) ; mult. by diffuse albedo

Specular Term (Phong) Specular Term (Blinn-Phong)

Sum over all light sources L (x,ω ) = L (x,ω ) + L (ω i n)s r r e r ∑ i h Reflected Light Emission Incident Blinn-Phong model (Output Image) Light (from (using half-angle) light source) (s is shininess) n ωh ωωio+ ωh = ||ωωio+

ω ωr i x

4 Outline of Lecture Syllabus / Taxonomy: Basic Hardware

§. Basics of hardware pipeline §. Basic Hardware Techniques (next wk) §. Shadow Mapping §. Reflection and Rendering equations §. Environment Mapping §. Typical Lighting, shading in hardware §. Graphics Hardware §. Taxonomy of methods/papers §. Geometry Engine (82) §. Reality Engine (83) §. Realistic Hardware-Accelerated Shading and Lighting §. Assignment: Sign up for paper presentations §. Real-Time Procedural Shading (01) §. Ray Tracing on Graphics Hardware (02) §. Brook: Stream Computing (04) Basic concepts/review only. If interested in more §. Photon mapping on hardware (03) background/in depth detail, refer to handouts §. GPU-Based Interactive Global Illumination (09)

High Quality Hardware Shading General Programmable Shading §. First programmable shading systems for hardware §. Heidrich 99, one of first papers. Hardware §. Peercy et al. 00: Multipass OpenGL implementation of tricks (before current programmable RenderMan (OpenGL as general SIMD machine) graphics; in standard OpenGL) §. Purcell et al. 02, map ray tracing to standard §. Proudfoot et al. 01: Multiple computation frequencies programmable hardware scanline pipeline (precursor to vertex, fragment shaders) §. Mark et al. 03: Describes nVidia’s Cg: one of first commercial high-level shading languages

§. Lindholm et al. 01, first nVidia vertex shaders (GeForce 3)

Shadow Mapping Syllabus/Taxonomy

§. Classic technique to add complex curved shadows §. Hardware Ray Tracing §. Williams 78, oldest paper we read in course §. Multi-Level CPU Ray Tracing (05) §. Many recent extensions for programmable hardware §. OptiX (10) §. Soft shadows: Agrawala 00 §. Embree (14) §. Complex geometry like hair: Lokovic and Veach 00 §. Adaptive techniques: Fernando 01, Stamminger 02, Sen 03 §. Image-Based Rendering (background) §. Light Fields and Lumigraphs (96) §. Surface Light Fields (00) §. Reflectance Fields (00) §. Signal Processing (background) §. Plenoptic Sampling (00) §. Signal-Processing Framework (01) §. Frequency Analysis of Light Transport (05)

5 New Ideas Precomputed Transfer

§. Precomputed Rendering §. Precompute on static scenes, followed by real-time relighting, §. Precomputed Radiance Transfer (02) changing view. Can capture most complex shading effects. §. Clustered PCA (03) §. §. All-Frequency Shadows (03) Sloan et al. 02,03: Low-frequency SH, compression §. Newer Papers (09, 11, 13) §. Ng et al. 03, 04: Wavelets all-frequency, relight, view change §. Sparse Sampling and Reconstruction §. Sloan 04, Wang 04: Factored BRDFs all-frequency relighting §. (background) (08,09,12) §. Axis-Aligned Filtering (12,13) §. Sampling of Recent Work §. Anisotropic Spherical Gaussians (13) §. Convolution Shadows, Volumes (13) §. Axis-Aligned Distribution Effects (14) §. Interreflections all-frequency BRDFs (14) §. Dynamic ray stream traversal (14) §. Machine learning filtering noise (15)

Outline of Lecture Paper Presentations

§. Basics of hardware pipeline §. E-mail me the top 3-5 papers you want to present §. Reflection and Rendering equations §. Only those that say “presented by students” §. Typical Lighting, shading in hardware §. Brief description of projects next week (see schedule) §. Taxonomy of methods/papers §. Project milestone and final proposal 3 weeks after

§. Assignment: Sign up for paper presentations

Basic concepts/review only. If interested in more background/in depth detail, refer to handouts

Questions?

6