INFOMAGR – Advanced Graphics Jacco Bikker - November 2016 - February 2017 Lecture 6 - “Light Transport” Welcome! 푰 풙, 풙′ = 품(풙, 풙′) 흐 풙, 풙′ + 흆 풙, 풙′, 풙′′ 푰 풙′, 풙′′ 풅풙′′ 푺 Today’s Agenda:

. Introduction . The Rendering Equation . Light Transport Advanced Graphics – Light Transport 3 Introduction

Whitted Advanced Graphics – Light Transport 4 Introduction

Whitted Advanced Graphics – Light Transport 5 Introduction

Whitted

Missing:

. Area lights . Glossy reflections . Caustics . Diffuse interreflections . Diffraction . . Phosphorescence . Temporal effects . Motion blur . Depth of field . Anti-aliasing Advanced Graphics – Light Transport 6 Introduction

Anti-aliasing

Adding anti-aliasing to a Whitted-style ray tracer:

Send multiple primary rays through each pixel, and average their result.

Problem:

. How do we aim those rays? . What if all rays return the same color? Advanced Graphics – Light Transport 7 Introduction

Anti-aliasing – Sampling Patterns

Adding anti-aliasing to a Whitted-style ray tracer:

Send multiple primary rays through each pixel, and average their result.

Problem:

. How do we aim those rays? . What if all rays return the same color? Advanced Graphics – Light Transport 8 Introduction

Anti-aliasing – Sampling Patterns Advanced Graphics – Light Transport 9 Introduction

Anti-aliasing – Sampling Patterns Advanced Graphics – Light Transport 10 Introduction

Anti-aliasing – Sampling Patterns

Adding anti-aliasing to a Whitted-style ray tracer:

Send multiple primary rays through each pixel, and average their result.

Problem:

. How do we aim those rays? . What if all rays return the same color? Advanced Graphics – Light Transport 11 Introduction

Whitted

Missing:

. Area lights . Glossy reflections . Caustics . Diffuse interreflections . Diffraction . Polarization . Phosphorescence . Temporal effects . Motion blur . Depth of field  Anti-aliasing Advanced Graphics – Light Transport 12 Introduction

Distribution *

Soft shadows

*: , Cook et al., 1984 Advanced Graphics – Light Transport 13 Introduction

Distribution Ray Tracing*

Glossy reflections

*: Distributed Ray Tracing, Cook et al., 1984 Advanced Graphics – Light Transport 14 Introduction

Distribution Ray Tracing*

?

*: Distributed Ray Tracing, Cook et al., 1984 Advanced Graphics – Light Transport 15 Introduction

Distribution Ray Tracing*

*: Distributed Ray Tracing, Cook et al., 1984 Advanced Graphics – Light Transport 16 Introduction

Distribution Ray Tracing

Whitted-style ray tracing is a point sampling algorithm:

. We may miss small features . We cannot sample areas

Area sampling:

. Anti-aliasing: one pixel . Soft shadows: one area light source . Glossy reflection: directions in a cone . Diffuse reflection: directions on the hemisphere Advanced Graphics – Light Transport 17 Introduction

Area Lights

Visibility of an area light source:

푉퐴 = 푉 푥, ꙍ𝑖 푑ꙍ𝑖 퐴

Analytical solution case 1:

푉퐴 = 퐴푙𝑖𝑔ℎ푡 − 퐴푙𝑖𝑔ℎ푡⋂푠푝ℎ푒푟푒

Analytical solution case 2:

푉퐴 = ? Advanced Graphics – Light Transport 18 Introduction

Approximating Integrals

An integral can be approximated as a Riemann sum:

퐵 푁 푁 푉퐴 = 푓(푥) 푑푥 ≈ 푓 푡𝑖 훥𝑖 , where 훥𝑖 = 퐵 − 퐴 Image from Wikipedia 퐴 𝑖=1 𝑖=1

Note that the intervals do not need to be uniform, as long as we sample the full interval. If the intervals are uniform, then

푁 푁 푁 퐵 − 퐴 푓 푡 훥 = 훥 푓 푡 = 푓 푡 . 𝑖 𝑖 𝑖 𝑖 푁 𝑖 𝑖=1 𝑖=1 𝑖=1

Regardless of uniformity, restrictions apply to 푁 when sampling multi-dimensional functions (ideally, 푁 = 푀푑). Also note that aliasing may occur if the intervals are uniform. Advanced Graphics – Light Transport 19 Introduction

Monte Carlo Integration

Alternatively, we can approximate an integral by taking random samples: 푁 퐵 퐵 − 퐴 푉 = 푓(푥) 푑푥 ≈ 푓 x Image from Wikipedia 퐴 푁 𝑖 퐴 𝑖=1

Here, x 1. . x 푁 ∈ [퐴, 퐵]. As 푁 approaches infinity, 푉퐴 approaches the expected value of 푓.

Unlike in Riemann sums, we can use arbitrary 푁 for Monte Carlo integration, regardless of dimension. Advanced Graphics – Light Transport 20 Introduction

Monte Carlo Integration of Area Light Visibility

To estimate the visibility of an area light source, we take 푁 random point samples.

In this case, 5 out of 6 samples are unoccluded:

1 5 푉 ≈ 1 + 1 + 1 + 0 + 1 + 1 = 6 6

In terms of Monte Carlo integration: 푁 1 푉 = 푉(푝) 푑푝 ≈ 푉 푝 2 푁 풮 𝑖=1

With a small number of samples, the variance in the estimate shows up as noise in the image. Advanced Graphics – Light Transport 21 Introduction

Monte Carlo Integration of Area Light Visibility

We can also use Monte Carlo to estimate the contribution of multiple lights:

1. Take the average of N samples from each light source; 2. Sum the averages.

2

퐸 푥 ← = 퐿𝑖 푉(푥 ↔ 푙𝑖) 𝑖=1

푥 Advanced Graphics – Light Transport 22 Introduction

Monte Carlo Integration of Area Light Visibility

Alternatively, we can just take 푁 samples, and pick a random light source for each sample.

푁 2 퐸 푥 ← = 퐿 푉 푝 , 푞 ∈ {1,2} 푁 푞 푞 𝑖=1 푁 1 퐿푞 푉푞 푝 = 푁 0.5 𝑖=1

푥 Advanced Graphics – Light Transport 23 Introduction

Monte Carlo Integration of Area Light Visibility

We obtain a better estimate with fewer samples if we do not treat each light equally.

In the previous example, each light had a 50% probability of being sampled. We can use an arbitrary probability, by dividing the sample by this probability.

푁 1 퐿푞 푉푞 푝 퐸 푥 ← = , 휌푞 = 1, 휌푞 > 0 푁 휌푞 𝑖=1

푥 Advanced Graphics – Light Transport 24 Introduction

Distribution Ray Tracing

Key concept of distribution ray tracing: We estimate integrals using Monte Carlo integration.

Integrals in rendering:

. Area of a pixel . Lens area (aperture) . Frame time . Light source area . Cones for glossy reflections . . … Advanced Graphics – Light Transport 25 Introduction

Open Issues

Remaining issues:

. Energy distribution in the ray tree / efficiency . Diffuse interreflections Today’s Agenda:

. Introduction . The Rendering Equation . Light Transport Advanced Graphics – Light Transport 27 Rendering Equation

Whitted, Cook & Beyond

Missing in Whitted: Cook:

. Area lights  Area lights . Glossy reflections  Glossy reflections . Caustics × Caustics . Diffuse interreflections × Diffuse interreflections . Diffraction × Diffraction . Polarization × Polarization . Phosphorescence × Phosphorescence . Temporal effects × Temporal effects . Motion blur  Motion blur . Depth of field  Depth of field . Anti-aliasing  Anti-aliasing Advanced Graphics – Light Transport 28 Rendering Equation

Whitted, Cook & Beyond

Cook’s solution to rendering:

Sample the many-dimensional integral using Monte Carlo integration.

… 퐴푝𝑖푥푒푙 퐴푙푒푛푠 푇푓푟푎푚푒 훺푔푙표푠푠푦 퐴푙𝑖푔ℎ푡

Ray are still used for specular reflections and refractions: The ray tree is not eliminated.

(In fact: for each light, one or more shadow rays are produced) Advanced Graphics – Light Transport 29 Rendering Equation

God’s Algorithm

1 room 1 bulb 100 watts 1020 per second

Photon behavior:

. Travel in straight lines . Get absorbed, or change direction: . Bounce (random / deterministic) . Get transmitted . Leave into the void . Get detected Advanced Graphics – Light Transport 30 Light Transport Advanced Graphics – Light Transport 31 Rendering Equation

God’s Algorithm - Mathematically

A may arrive at a sensor after travelling in a straight line from a light source to the sensor:

퐿 푠 ← 푥 = 퐿퐸(푠 ← 푥)

Or, it may be reflected by a surface towards the sensor:

′ ′ ′ ′ 퐿 푠 ← 푥 = 푓푟 푠 ← 푥 ← 푥 퐿 푥 ← 푥 퐺 푥 ↔ 푥 푑퐴(푥 ) 퐴

Those are the options. Adding direct and indirect illumination together:

′ ′ 퐿 푠 ← 푥 = 퐿퐸 푠 ← 푥 + 푓푟 푠 ← 푥 ← 푥′ 퐿 푥 ← 푥′ 퐺 푥 ↔ 푥 푑퐴(푥 ) 퐴 Advanced Graphics – Light Transport 32 Rendering Equation

God’s Algorithm - Mathematically

′ ′ 퐿 푠 ← 푥 = 퐿퐸 푠 ← 푥 + 푓푟 푠 ← 푥 ← 푥′ 퐿 푥 ← 푥′ 퐺 푥 ↔ 푥 푑퐴(푥 ) 퐴

Geometry factor Indirect

Reflection

Hemisphere

Emission Advanced Graphics – Light Transport 33 Rendering Equation

′ ′ 퐿 푠 ← 푥 = 퐿퐸 푠 ← 푥 + 푓푟 푠 ← 푥 ← 푥′ 퐿 푥 ← 푥′ 퐺 푥 ↔ 푥 푑퐴(푥 ) 퐴

The Rendering Equation:*

. Light transport from lights to sensor . Physically based

The equation allows us to determine to which extend rendering algorithms approximate real-world light transport.

*: The Rendering Equation, Kajiya, 1986 Advanced Graphics – Light Transport 34 Rendering Equation

to the Rendering Equation:

′ ′ Light sourc퐿 푠 ← 푥 = 퐿퐸 푠 ← 푥 + 퐴 푓푟 푠 ← 푥 ← 푥′ 퐿 푥 ← 푥′ 퐺 푥 ↔ 푥 푑퐴(푥 )

Rasterization, according es limited to 푁 point lights . Visibility handled by z-buffer / Painter’s algorithm . Visibility not supported for lights

푁퐿

퐿 푠 ← 푥 = 퐿퐸 푠 ← 푥 + 푓푟 푠 ← 푥 ← 퐿𝑖 퐿𝑖 퐺 푥 ↔ 퐿𝑖 𝑖=1

(note: this does not take into account approximations such as shadow maps and environment maps) Advanced Graphics – Light Transport 35 Rendering Equation

′ ′ 퐿 푠 ← 푥 = 퐿퐸 푠 ← 푥 + 푓푟 푠 ← 푥 ← 푥′ 퐿 푥 ← 푥′ 퐺 푥 ↔ 푥 푑퐴(푥 ) 퐴

Whitted-style ray tracing, according to the Rendering Equation:

. Inter-surface reflections limited to specular surfaces . Light sources limited to 푁 point lights . Visibility is supported

푁퐿

퐿 푠 ← 푥 = 퐿퐸 푠 ← 푥 + 푓푟 푠 ← 푥 ← 퐿𝑖 퐿𝑖 퐺 푥 ↔ 퐿𝑖 𝑖=1

훿 ′ ′ + 푓푟 푠 ← 푥 ← 푥′ 퐿 푥 ← 푥′ 퐺 푥 ↔ 푥 푑퐴(푥 ) 퐴

(note: only specular recursive transport supported) Today’s Agenda:

. Introduction . The Rendering Equation . Light Transport Advanced Graphics – Light Transport 37 Light Transport

′ ′ 퐿 푠 ← 푥 = 퐿퐸 푠 ← 푥 + 푓푟 푠 ← 푥 ← 푥′ 퐿 푥 ← 푥′ 퐺 푥 ↔ 푥 푑퐴(푥 ) 퐴

Relation between real-world light transport and the RE:

1. Each sensor element registers an amount of photons arriving from the first surface visible though that pixel. 2. This surface may be emissive, in which case it produced the sensed photons. 3. This surface may also reflect photons, arriving from other surfaces in the scene. 4. For the other surfaces: goto 2. Advanced Graphics – Light Transport 38 Light Transport

Light Transport Quantities

Radiant flux - 훷 :

“Radiant energy emitted, reflected, transmitted or received, per unit time.”

Units: watts = joules per second 푊 = 퐽 푠−1 .

Simplified particle analogy: number of photons.

Note: photon energy depends on electromagnetic : hc E = , where h is Planck’s constant, c is the speed of light, λ and λ is wavelength. At λ = 550nm (yellow), a single photon carries 3.6 ∗ 10−19 joules. Advanced Graphics – Light Transport 39 Light Transport

Light Transport Quantities

In a vacuum, radiant flux emitted by a point light source remains constant over distance:

A point light emitting 100W delivers 100W to the surface of a sphere of radius r around the light. This sphere has an area of 휋푟2; energy per surface area thus decreases by 1/푟2.

In terms of photons: the density of the photon distribution decreases by 1/푟2. Advanced Graphics – Light Transport 40 Light Transport

Light Transport Quantities

A surface receives an amount of light energy proportional to its solid angle: the two-dimensional space that an object subtends at a point.

Solid angle units: steradians (sr).

Corresponding concept in 2D: radians; the length of the arc on the unit sphere subtended by an angle. Advanced Graphics – Light Transport 41 Light Transport

Light Transport Quantities

Radiance - 퐿 :

“The power of electromagnetic radiation 퐿 emitted, reflected, transmitted or received per unit projected area per unit solid angle.”

Units: 푊푠푟−1푚−2

Simplified particle analogy: Amount of particles passing through a pipe with unit diameter, per unit time.

Note: radiance is a continuous value: while flux at a point is 0 (since both area and solid angle are 0), we can still define flux per area per solid angle for that point. Advanced Graphics – Light Transport 42 Light Transport

Light Transport Quantities

Irradiance - 퐸 :

“The power of electromagnetic radiation per unit area incident on a surface.” 푁 Units: Watts per 푚2 = joules per second per 푚2 푊푚−2 = 퐽푚−2푠−1 .

Simplified particle analogy: number of photons arriving per unit area per unit time, from all directions. Advanced Graphics – Light Transport 43 Light Transport

Light Transport Quantities 퐿 Converting radiance to :

퐸 = 퐿 cos 휃

퐿 푁 휃 Advanced Graphics – Light Transport 44 Light Transport

Pinhole Camera

A camera should not accept light from all directions for a particular pixel on the film. A pinhole ensures that only a single direction is sampled.

In the real world, an aperture with a lens is used to limit directions to a small range, but only on the focal plane. Advanced Graphics – Light Transport 45 Light Transport

Light Transport

′ ′ 퐿 푠 ← 푥 = 퐿퐸 푠 ← 푥 + 푓푟 푠 ← 푥 ← 푥′ 퐿 푥 ← 푥′ 퐺 푥 ↔ 푥 푑퐴(푥 ) 퐴

퐿표 푥, 휔표 = 퐿퐸 푥, 휔표 + 푓푟 푥, 휔표, 휔𝑖 퐿𝑖 푥, 휔𝑖 cos 휃𝑖 푑휔𝑖 훺 Radiance Radiance Radiance

Irradiance

BRDF Advanced Graphics – Light Transport 50 Light Transport

Flux / Particle Count / Probability

So far, we measured light transport as energy:

Radiance = Joules per second per square meter; Irradiance = Radiance projected to the surface.

Alternatively, we can imagine this as a large number of particles (say, 1020). How does this affect the concept of radiance, irradiance and BRDF?

Alternatively, we can think of irradiance and the BRDF as a probability distribution. How does this affect the concept of irradiance and the BRDF? Today’s Agenda:

. Introduction . The Rendering Equation . Light Transport INFOMAGR – Advanced Graphics Jacco Bikker - November 2016 - February 2017

END of “Light Transport”

next lecture: “”