Chapter 7

Signals and Sampling 7.1 Sampling Theory 7.2 Image Sampling Interface

Chapter 7 of “Physically Based 7.3 Stratified Sampling Rendering” by Pharr&Humphreys 7.4 Low-Discrepancy Sampling 7.5 Best-Candidate Sampling Patterns 7.6 Image Reconstruction

2

Additional Reading Motivation Chapter 14.10 of “CG: Principles & Practice” by • Real World - continuous Foley, van Dam et al. • Digital (Computer) world - discrete Chapter 4, 5, 8, 9, 10 in “Principles of Digital Image Synthesis,” by A. Glassner • Typically we have to either: Chapter 4, 5, 6 of “Digital Image Warping” by – create discrete data from continuous or (e.g. Wolberg rendering/ray-tracing, illumination models, Chapter 2, 4 of “Discrete-Time Signal Processing” by morphing) Oppenheim, Shafer – manipulate discrete data (textures, surface description, image processing,tone mapping)

Motivation Motivation

• Artifacts occurring in sampling - aliasing: Engineering approach: – Jaggies nearest neighbor: – Moire Convolution with – Flickering small objects box filter – Sparkling highlights – Temporal strobing linear filter:

• Preventing these artifacts - Antialiasing Convolution with tent filter Motivation- Graphics Motivation

Engineering approach: Original “manipulated” • black-box (continuous) signal “Graphics” (continuous) signal “System” or Algorithm • discretization: sampling Reconstruction sampled Multiplication with filter “shah” function signal

Convolution Convolution (2)

• How can we characterize our “black box”? • Impulse: • We assume to have a “nice” box/algorithm: – linear – time-invariant • discrete impulse: • then it can be characterized through the response to an “impulse”: • Finite Impulse Response (FIR) vs. “System” or • Infinite Impulse Response (IIR) Algorithm

Convolution (3) Convolution (4) • for a time-invariant system h[k-n] would • An arbitrary signal x[k] can be written as: be the impulse response to a delayed impulse d[k-n] • Let the impulse response be h[k]: • hence, if y[k] is the response of our system to the input x[k] (and we assume a linear system): “System” or ![k] h[k] IIR - N=inf. Algorithm FIR - N

“System” or x[k] y[k] Algorithm Fourier Transforms Fourier Transforms (2)

• Let’s look at a special input sequence: • Hence is an eigen-function and H(") its eigenvalue • then: • H(") is the Fourier-Transform of the h[n] and hence characterizes the underlying system in terms of frequencies • H(") is periodic with period 2# • H(") is decomposed into – phase (angle) response – magnitude response

Properties Properties (2)

• Linear • Parseval’s Theorem • scaling • convolution • Multiplication • preserves “Energy” - overall signal content • Differentiation

• delay/shift

Transforms Pairs Transform Pairs - Shah

• Sampling = Multiplication with a Shah Fourier function: Transform sampling Average T 1/T Filter • multiplication in spatial domain = convolution in the frequency domain Box/Sinc • frequency replica of primary spectrum Filter (also called aliased spectra) Transforms Pairs (2) General Process

Original function Linear Sampled function Acquisition Filter

Gaussian Reconstructed Filter Reconstruction Function Re-sampled function Resampling derivative Filter

How? - Reconstruction Sampling Theorem

• A signal can be reconstructed from its Spatial Domain: Frequency Domain: samples without loss of information if the original signal has no frequencies above 1/2 of the sampling frequency • For a given bandlimited function, the rate at Mathematically: which it must be sampled is called the • Convolution: f(x)*h(x) • Multiplication: Nyquist frequency

Evaluated at discrete online demo points (sum)

Example Example

2D 1D Given Given

Needed Needed Nearest neighbor Linear Interpolation General Process - Pre-Filtering Frequency Domain

Original function Sampled function Original function Band-limited function Acquisition Pre-Filtering

Reconstructed Sampled Acquisition Reconstruction Function Function Re-sampled function Reconstructed function Resampling Reconstruction

Once Again ... Pipeline - Example

Spatial domain Frequency domain

Pre-filter

Post-aliasing x *

Pre-aliasingsampling Reconstruction filter * x

Pipeline - Example (2) Pipeline - Example (3)

Spatial domain Frequency domain Spatial domain Frequency domain

* x * x

x * Sources of Aliasing Aliasing

• Non-bandlimited signal sampling

• Low sampling rate (below Nyquist)

sampling

• Non perfect reconstruction

Bandlimited Interpolation

Spatial Domain: Frequency Domain: • convolution is exact • cut off freq. replica

Derivatives Reconstruction Kernels

Spatial Domain: Frequency Domain: • Nearest Neighbor (Box) • convolution is exact • cut off freq. replica

• Linear

• Sinc

• Gaussian • Many others Spatial d. Frequency d. Ideal Reconstruction Ideal Reconstruction

• Box filter in frequency domain = • Use the sinc function – to bandlimit the • Sinc Filter in spatial domain sampled signal and remove all copies of the • impossible to realize (really?) spectra introduced by sampling Pass-band stop- • But: band – The sinc has infinite extent and we must use Ideal filter simpler filters with finite extents. Smoothing – The windowed versions of sinc may introduce Practical ringing artifacts which are perceptually filter Post-aliasing objectionable.

Reconstructing with Sinc Ideal Reconstruction

– Realizable filters do not have sharp transitions; also have ringing in pass/stop bands

# # # Low-pass band-pass high-pass filter filter filter

Higher Dimensions? Possible Errors

• Design typically in 1D • Post-aliasing – reconstruction filter passes frequencies beyond the • extensions to higher dimensions (typically): Nyquist frequency (of duplicated frequency spectrum) – separable filters => frequency components of the original signal appear in the reconstructed signal at different frequencies – radially symmetric filters • Smoothing – limited results – frequencies below the Nyquist frequency are attenuated • research topic • Ringing (overshoot) T – occurs when trying to sample/reconstruct discontinuity • Anisotropy ? – caused by not spherically symmetric filters Aliasing vs. Noise Antialiasing

• Antialiasing = Preventing aliasing • 1. Analytically pre-filter the signal – Solvable for points, lines and polygons – Not solvable in general (e.g. procedurally defined images) • 2. Uniform supersampling and resample • 3. Nonuniform or stochastic sampling

Distribution of Extrafoveal Uniform Supersampling Cones • Increasing the sampling rate moves each • Yellot theory (1983) copy of the spectra further apart, potentially – Aliases replaced by noise reducing the overlap and thus aliasing – Visual system less sensitive to high freq noise • Resulting samples must be resampled Monkey eye cone distribution Fourier Transform (filtered) to image sampling rate

Non-Uniform Sampling - Non-Uniform Sampling - Intuition Patterns • Uniform sampling • Poisson – The spectrum of uniformly spaced samples is also a set – Pick n random points in sample space of uniformly spaced spikes – Multiplying the signal by the sampling pattern • Uniform Jitter corresponds to placing a copy of the spectrum at each – Subdivide sample space into n regions spike (in freq. space) – Aliases are coherent, and very noticeable • Poisson Disk • Non-uniform sampling – Pick n random points, but not too close – Samples at non-uniform locations have a different spectrum; a single spike plus noise – Sampling a signal in this way converts aliases into broadband noise – Noise is incoherent, and much less objectionable Poisson Disk Sampling Uniform Jittered Sampling

Spatial Domain Fourier Domain Spatial Domain Fourier Domain

Non-Uniform Sampling - Stratified Sampling Patterns • Spectral characteristics of these • Put at least one sample in each strata distributions: • Multiple samples in strata do no good – Poisson: completely uniform (white noise). • Also have samples far away from each High and low frequencies equally present other – Poisson disc: Pulse at origin (DC component of image), surrounded by empty ring (no low frequencies), surrounded by white noise • Graphics: jittering – Jitter: Approximates Poisson disc spectrum, but with a smaller empty disc.

Stratification Stratification

• OR – Split up the integration domain in N disjoint sub-domains or strata – Evaluate the integral in each of the sub- domains separately with one or more samples. • More precisely: More Jittered Sequences Jitter

• Place samples in the grid • Perturb the samples up to 1/2 width or height

Texture Example Multiple Dimensions

• Too many samples Exact – 256 samples/pixel Jitter with 1 sample/pixel • 1D • 2D 3D

1 sample/pixel Jitter with 4 samples/pixel

Jitter Problems Multiple Dimensions

• How to deal with higher dimensions? • Make (separate) strata for each dimension – Curse of dimensionality • Randomly associate strata among each – D dimensions means ND “cells” (if we use a other separable extension) • Ensure good sample “distribution” • Solutions: – Example: 2D screen position; 2D lens position; – We can look at each dimension independently 1D time – We can either look in non-separable geometries – Latin Hypercube (or N-Rook) sampling Optimal sampling lattices Optimal sampling lattices

• Dividing space up into equal cells doesn’t • We have to deal with different geometry have to be on a Cartesian lattices • 2D - hexagon • In fact - Cartesian is NOT the optimal way • 3D - truncated octahedron how to divide up space uniformly

Cartesian Hexagonal

Latin Hypercubes - N-Rooks Stratification - problems

• Distributing n samples in D dimensions, • Clamping (LHS helps) } even if n is not a power of D • Could still have large • Divide each dimension in n strata empty regions • Generate a jittered sample in each of the n diagonal entries • Other geometries, • Random shuffle in each dimension e.g. stratify circles or spheres?

How good are the samples ? Discrepancy

• How can we evaluate how well our samples • DN - Maximum difference between the are distributed? fraction of N points xi and relative size of – No “holes” volume [0,1]n – No clamping • Pick a set of V • • N points • Well distributed patterns have low sub-volumes B of [0,1]n • • • • • discrepancy • • • – Small = evenly distributed • • – Large = clustering • • • • • D ->0 when N is n points • • Construct low discrepancy sequence N • very large • • • • Discrepancy Discrepancy • How to create low-discrepancy sequences? • Examples of sub-volumes B of [0,1]d: – Deterministic sequences!! Not random – Axis-aligned anymore – Share a corner at the origin (star – Also called pseudo-random discrepancy) V – Advantage - easy to compute • N points • Best discrepancy that has • • 1D: • • • • • been obtained in d • dimensions: • • • • • • • • n points • • • • • •

Pseudo-Random Sequences Van Der Corput Sequence

• Radical inverse • Most simple sequence – Building block for high-D sequences • Uses radical inverse of base 2 i binary radical x – “inverts” an integer given in base b • Achieves minimal i form of i inverse possible discrepancy 0 0 0.0 0 1 1 0.1 0.5 2 10 0.01 0.25 3 11 0.11 0.75 0 4 2 1 3 4 100 0.001 0.125 5 101 0.101 0.625 6 110 0.011 0.375

Halton Hammersley Sequences • Can be used if N is not known in advance • All prefixes of a sequence are well • Similar to Halton distributed • Need to know total number of samples in • Use prime number bases for each advance dimension • Better discrepancy than Halton • Achieves best possible discrepancy Hammersley Sequences Hammersley Sequences

Folded Radical Inverse Examples

• Hammersley-Zaremba • Halton-Zaremba • Improves discrepancy

(t,m,d) nets (t,d) Sequences

• The most successful constructions of low- • (t,m,d)-Nets ensures, that all samples are discrepancy sequences are based on (t,m,d)- uniformly distributed for any integer subdivision nets and (t,d)-sequences. of our space. d • Basis b; • (t,d)-sequence is a sequence xi of points in [0,1] such that for all integers and m>t, the point • Is a point set in [0,1]d consisting of bm set points, such that every box is a (t,m,d)-net in base b. • The number t is the quality parameter. Smaller of volume bt-m contains bt points values of t yield more uniform nets and sequences because b-ary boxes of smaller volume still contain points. (0,2) Sequences Practical Issues

• Used in pbrt for the Low-discrepancy • Create one sequence sampler • Create new ones from the first sequence by • Base 2 “scrambling” rows and columns • This is only possible for (0,2) sequences, since they have such a nice property (the “n-rook” property)

Texture Best-Candidate Sampling

• Jittered stratification – Randomness (inefficient) – Clustering problems – Undersampling (“holes”)

Jitter with 1 sample/pixel • Low Discrepancy Sequences – Still (visibly) aliased • “Ideal”: Poisson disk distribution – too computationally expensive • Best Sampling - approximation to Poisson disk

Hammersley Sequence with 1 sample/pixel

Poisson Disk Best-Candidate Sampling

• Comes from structure of eye – rods and cones • Dart Throwing • No two points are closer than a threshold • Very expensive • Compromise – Best Candidate Sampling – Compute pattern which is reused by tiling the image plane (translating and scaling). Jittere Poisson Disk Best Candidate d – Toroidal topology – Effects the distance between points on top to bottom Best-Candidate Sampling Texture

Texture Next

Jitter with 1 sample/pixel Best Candidate with 1 sample/pixel • Probability Theory • Monte Carlo Techniques • Rendering Equation

Jitter with 4 sample/pixel Best Candidate with 4 sample/pixel