To Do

Computer Graphics II: Rendering § Submit homework 3 by Thu

CSE 168[Spr 20],Lecture 11: Fourier Analysis, Sampling § Start immediately on homework 4. Ravi Ramamoorthi § Start thinking about final project

http://viscomp.ucsd.edu/classes/cse168/sp20 § This lecture gives core background on sampling and signal-processing (bear in mind image processing)

Some slides courtesy Pat Hanrahan

Quality Improves with More Rays

Sampling and Reconstruction § An image is a 2D array of samples § Discrete samples from real-world continuous signal

1 Sampling and Reconstruction (Spatial) Aliasing

(Spatial) Aliasing Sampling a Zone Plate

§ Jaggies probably biggest aliasing problem

Sampling and Aliasing Image Processing pipeline

§ Artifacts due to undersampling or poor reconstruction § Formally, high frequencies masquerading as low § E.g. high frequency line as low freq jaggies

2 Motivation Ideas

§ Formal analysis of sampling and reconstruction § Signal (function of time generally, here of space) § Important theory (signal-processing) for graphics § Continuous: defined at all points; discrete: on a grid § Also relevant in rendering, modeling, animation § High frequency: rapid variation; Low Freq: slow variation § Images are converting continuous to discrete. Do this § Note: Fourier Analysis useful for understanding, sampling as best as possible. but image processing often done in spatial domain § Signal processing theory tells us how best to do this § Based on concept of frequency domain Fourier analysis

Sampling Theory Fourier Transform Analysis in the frequency (not spatial) domain § § Sum of sine waves, with possibly different offsets (phase) Tool for converting from spatial to frequency domain § Each wave different frequency, amplitude +∞ f (x) = ∑ F(u)e2πiux u=−∞ e2πiux = cos(2πux) + i sin(2πux) § Or vice versa i = −1 § One of most important mathematical ideas § Computational algorithm: Fast Fourier Transform § One of 10 great algorithms scientific computing § Makes Fourier processing possible (images etc.) § Not discussed here, but look up if interested

Fourier Transform Fourier Transform

§ Simple case, function sum of sines, cosines § Simple case, function sum of sines, cosines +∞ +∞ f (x) = ∑ F(u)e2πiux f (x) = ∑ F(u)e2πiux u=−∞ u=−∞ 1 1 F(u) = f (x)e−2πiuxdx F(u) = f (x)e−2πiuxdx ∫0 ∫0 § Continuous infinite case § Discrete case ∞ x=N−1 −2πiux ⎡ ⎤ Forward Transform: F(u) = f (x)e dx F(u) = ∑ f (x)⎣cos(2πux / N) − i sin(2πux / N)⎦, 0 ≤ u ≤ N −1 ∫−∞ x=0 +∞ u=N−1 2πiux 1 ⎡ ⎤ Inverse Transform: f (x) = F(u)e du f (x) = ∑ F(u)⎣cos(2πux / N) + i sin(2πux / N)⎦, 0 ≤ x ≤ N −1 ∫−∞ N u=0

3 Fourier Transform: Examples 1 Fourier Transform Examples 2 ∞ −2πiux Forward Transform: F(u) = f (x)e dx Single sine curve ∫−∞ +∞ (+constant DC term) 2πiux Inverse Transform: f (x) = F(u)e du ∫−∞ § Common examples

f (x) F(u)

−2πiux0 δ(x − x0 ) e

+∞ 1 δ(u) 2πiux f (x) = F(u)e 2 2 2 ∑ −ax π −π u /a u=−∞ e e 1 a F(u) = f (x)e−2πiuxdx ∫0

Fourier Transform Properties Sampling Theorem, Bandlimiting ∞ −2πiux Forward Transform: F(u) = f (x)e dx § A signal can be reconstructed from its samples, ∫−∞ if the original signal has no frequencies above +∞ 2πiux Inverse Transform: f (x) = F(u)e du half the sampling frequency – Shannon ∫−∞ § Common properties § The minimum sampling rate for a bandlimited § Linearity: F(af (x) + bg(x)) = aF(f (x)) + bF(g(x)) function is called the Nyquist rate

∞ § Derivatives: [integrate by parts] F(f '(x)) = f '(x)e−2πiuxdx ∫−∞ = 2πiuF(u) § ∞ 2D Fourier Transform ∞ −2πiux −2πivy Forward Transform: F(u,v) = f (x,y)e e dxdy ∫ ∫−∞ −∞ ∞ +∞ 2πiux 2πivy § Convolution (next)I nverse Transform: f (x,y) = F(u,v)e e dudv ∫ ∫−∞ −∞

Sampling Theorem, Bandlimiting Antialiasing

§ A signal can be reconstructed from its samples, if § Sample at higher rate the original signal has no frequencies above half § Not always possible the sampling frequency – Shannon § Real world: lines have infinitely high frequencies, can’t sample at high enough resolution § The minimum sampling rate for a bandlimited § Prefilter to bandlimit signal function is called the Nyquist rate § Low-pass filtering (blurring) § A signal is bandlimited if the highest frequency is § Trade blurriness for aliasing bounded. This frequency is called the bandwidth § In general, when we transform, we want to filter to bandlimit before sampling, to avoid aliasing

4 Ideal bandlimiting filter Convolution 1

§ Formal derivation is homework exercise

if full width fmax = 1

Convolution 2 Convolution 3

Convolution 4 Convolution 5

5 Convolution in Frequency Domain Practical Image Processing ∞ −2πiux § Forward Transform: F(u) = f (x)e dx Discrete convolution (in spatial domain) with filters for ∫−∞ various digital signal processing operations +∞ 2πiux § Inverse Transform: f (x) = F(u)e du Easy to analyze, understand effects in frequency domain ∫−∞ § E.g. blurring or bandlimiting by convolving with low pass filter § Convolution (f is signal ; g is filter [or vice versa]) +∞ +∞ h(y) = ∫ f (x)g(y − x)dx = ∫ g(x)f (y − x)dx −∞ −∞ h = f * g or f ⊗ g § Fourier analysis (frequency domain multiplication) H(u) = F(u)G(u)

Point vs Area Sampling Uniform Supersampling

Jittered Sampling

6 Jittered vs Uniform Supersampling Distribution of Extrafoveal Cones

Poisson Disk Sampling

7