Chapter 7 Signals and Sampling 7.1 Sampling Theory 7.2 Image Sampling Interface Chapter 7 of “Physically Based 7.3 Stratified Sampling Rendering” by Pharr&Humphreys 7.4 Low-Discrepancy Sampling 7.5 Best-Candidate Sampling Patterns 7.6 Image Reconstruction 2 Additional Reading Motivation Chapter 14.10 of “CG: Principles & Practice” by • Real World - continuous Foley, van Dam et al. • Digital (Computer) world - discrete Chapter 4, 5, 8, 9, 10 in “Principles of Digital Image Synthesis,” by A. Glassner • Typically we have to either: Chapter 4, 5, 6 of “Digital Image Warping” by – create discrete data from continuous or (e.g. Wolberg rendering/ray-tracing, illumination models, Chapter 2, 4 of “Discrete-Time Signal Processing” by morphing) Oppenheim, Shafer – manipulate discrete data (textures, surface description, image processing,tone mapping) Motivation Motivation • Artifacts occurring in sampling - aliasing: Engineering approach: – Jaggies nearest neighbor: – Moire Convolution with – Flickering small objects box filter – Sparkling highlights – Temporal strobing linear filter: • Preventing these artifacts - Antialiasing Convolution with tent filter Motivation- Graphics Motivation Engineering approach: Original “manipulated” • black-box (continuous) signal “Graphics” (continuous) signal “System” or Algorithm • discretization: sampling Reconstruction sampled Multiplication with filter “shah” function signal Convolution Convolution (2) • How can we characterize our “black box”? • Impulse: • We assume to have a “nice” box/algorithm: – linear – time-invariant • discrete impulse: • then it can be characterized through the response to an “impulse”: • Finite Impulse Response (FIR) vs. “System” or • Infinite Impulse Response (IIR) Algorithm Convolution (3) Convolution (4) • for a time-invariant system h[k-n] would • An arbitrary signal x[k] can be written as: be the impulse response to a delayed impulse d[k-n] • Let the impulse response be h[k]: • hence, if y[k] is the response of our system to the input x[k] (and we assume a linear system): “System” or ![k] h[k] IIR - N=inf. Algorithm FIR - N<inf. “System” or x[k] y[k] Algorithm Fourier Transforms Fourier Transforms (2) • Let’s look at a special input sequence: • Hence is an eigen-function and H(") its eigenvalue • then: • H(") is the Fourier-Transform of the h[n] and hence characterizes the underlying system in terms of frequencies • H(") is periodic with period 2# • H(") is decomposed into – phase (angle) response – magnitude response Properties Properties (2) • Linear • Parseval’s Theorem • scaling • convolution • Multiplication • preserves “Energy” - overall signal content • Differentiation • delay/shift Transforms Pairs Transform Pairs - Shah • Sampling = Multiplication with a Shah Fourier function: Transform sampling Average T 1/T Filter • multiplication in spatial domain = convolution in the frequency domain Box/Sinc • frequency replica of primary spectrum Filter (also called aliased spectra) Transforms Pairs (2) General Process Original function Linear Sampled function Acquisition Filter Gaussian Reconstructed Filter Reconstruction Function Re-sampled function Resampling derivative Filter How? - Reconstruction Sampling Theorem • A signal can be reconstructed from its Spatial Domain: Frequency Domain: samples without loss of information if the original signal has no frequencies above 1/2 of the sampling frequency • For a given bandlimited function, the rate at Mathematically: which it must be sampled is called the • Convolution: f(x)*h(x) • Multiplication: Nyquist frequency Evaluated at discrete online demo points (sum) Example Example 2D 1D Given Given Needed Needed Nearest neighbor Linear Interpolation General Process - Pre-Filtering Frequency Domain Original function Sampled function Original function Band-limited function Acquisition Pre-Filtering Reconstructed Sampled Acquisition Reconstruction Function Function Re-sampled function Reconstructed function Resampling Reconstruction Once Again ... Pipeline - Example Spatial domain Frequency domain Pre-filter Post-aliasing x * Pre-aliasingsampling Reconstruction filter * x Pipeline - Example (2) Pipeline - Example (3) Spatial domain Frequency domain Spatial domain Frequency domain * x * x x * Sources of Aliasing Aliasing • Non-bandlimited signal sampling • Low sampling rate (below Nyquist) sampling • Non perfect reconstruction Bandlimited Interpolation Spatial Domain: Frequency Domain: • convolution is exact • cut off freq. replica Derivatives Reconstruction Kernels Spatial Domain: Frequency Domain: • Nearest Neighbor (Box) • convolution is exact • cut off freq. replica • Linear • Sinc • Gaussian • Many others Spatial d. Frequency d. Ideal Reconstruction Ideal Reconstruction • Box filter in frequency domain = • Use the sinc function – to bandlimit the • Sinc Filter in spatial domain sampled signal and remove all copies of the • impossible to realize (really?) spectra introduced by sampling Pass-band stop- • But: band – The sinc has infinite extent and we must use Ideal filter simpler filters with finite extents. Smoothing – The windowed versions of sinc may introduce Practical ringing artifacts which are perceptually filter Post-aliasing objectionable. Reconstructing with Sinc Ideal Reconstruction – Realizable filters do not have sharp transitions; also have ringing in pass/stop bands # # # Low-pass band-pass high-pass filter filter filter Higher Dimensions? Possible Errors • Design typically in 1D • Post-aliasing – reconstruction filter passes frequencies beyond the • extensions to higher dimensions (typically): Nyquist frequency (of duplicated frequency spectrum) – separable filters => frequency components of the original signal appear in the reconstructed signal at different frequencies – radially symmetric filters • Smoothing – limited results – frequencies below the Nyquist frequency are attenuated • research topic • Ringing (overshoot) T – occurs when trying to sample/reconstruct discontinuity • Anisotropy ? – caused by not spherically symmetric filters Aliasing vs. Noise Antialiasing • Antialiasing = Preventing aliasing • 1. Analytically pre-filter the signal – Solvable for points, lines and polygons – Not solvable in general (e.g. procedurally defined images) • 2. Uniform supersampling and resample • 3. Nonuniform or stochastic sampling Distribution of Extrafoveal Uniform Supersampling Cones • Increasing the sampling rate moves each • Yellot theory (1983) copy of the spectra further apart, potentially – Aliases replaced by noise reducing the overlap and thus aliasing – Visual system less sensitive to high freq noise • Resulting samples must be resampled Monkey eye cone distribution Fourier Transform (filtered) to image sampling rate Non-Uniform Sampling - Non-Uniform Sampling - Intuition Patterns • Uniform sampling • Poisson – The spectrum of uniformly spaced samples is also a set – Pick n random points in sample space of uniformly spaced spikes – Multiplying the signal by the sampling pattern • Uniform Jitter corresponds to placing a copy of the spectrum at each – Subdivide sample space into n regions spike (in freq. space) – Aliases are coherent, and very noticeable • Poisson Disk • Non-uniform sampling – Pick n random points, but not too close – Samples at non-uniform locations have a different spectrum; a single spike plus noise – Sampling a signal in this way converts aliases into broadband noise – Noise is incoherent, and much less objectionable Poisson Disk Sampling Uniform Jittered Sampling Spatial Domain Fourier Domain Spatial Domain Fourier Domain Non-Uniform Sampling - Stratified Sampling Patterns • Spectral characteristics of these • Put at least one sample in each strata distributions: • Multiple samples in strata do no good – Poisson: completely uniform (white noise). • Also have samples far away from each High and low frequencies equally present other – Poisson disc: Pulse at origin (DC component of image), surrounded by empty ring (no low frequencies), surrounded by white noise • Graphics: jittering – Jitter: Approximates Poisson disc spectrum, but with a smaller empty disc. Stratification Stratification • OR – Split up the integration domain in N disjoint sub-domains or strata – Evaluate the integral in each of the sub- domains separately with one or more samples. • More precisely: More Jittered Sequences Jitter • Place samples in the grid • Perturb the samples up to 1/2 width or height Texture Example Multiple Dimensions • Too many samples Exact – 256 samples/pixel Jitter with 1 sample/pixel • 1D • 2D 3D 1 sample/pixel Jitter with 4 samples/pixel Jitter Problems Multiple Dimensions • How to deal with higher dimensions? • Make (separate) strata for each dimension – Curse of dimensionality • Randomly associate strata among each – D dimensions means ND “cells” (if we use a other separable extension) • Ensure good sample “distribution” • Solutions: – Example: 2D screen position; 2D lens position; – We can look at each dimension independently 1D time – We can either look in non-separable geometries – Latin Hypercube (or N-Rook) sampling Optimal sampling lattices Optimal sampling lattices • Dividing space up into equal cells doesn’t • We have to deal with different geometry have to be on a Cartesian lattices • 2D - hexagon • In fact - Cartesian is NOT the optimal way • 3D - truncated octahedron how to divide up space uniformly Cartesian Hexagonal Latin Hypercubes - N-Rooks Stratification - problems • Distributing n samples in D dimensions, • Clamping (LHS helps) } even if n is not a power of D • Could still have large