Reading Required: Watt, Section 14.1 Recommended: Don P. Mitchell and Arun N. Netravali, “Reconstruction Filters in Computer Computer 2. Sampling theory Graphics ,” Computer Graphics, (Proceedings of SIGGRAPH 88). 22 (4), pp. 221-228, 1988. 1 2 What is an image? Images as functions We can think of an image as a function, f, from R2 to R: f( x, y ) gives the intensity of a channel at position ( x, y ) Realistically, we expect the image only to be defined over a rectangle, with a finite range: • f: [a,b]x[c,d] Æ [0,1] f(x,y) A color image is just three functions pasted together. y We can write this as a “vector-valued” function: x ªºrxy(, ) f (,xy )= «» gxy (, ) «» ¬¼«»bxy(, ) We’ll focus in grayscale (scalar-valued) images for now. 3 4 Digital images Motivation: filtering and resizing In computer graphics, we usually create or operate What if we now want to: on digital (discrete)images: smooth an image? Sample the space on a regular grid sharpen an image? Quantize each sample (round to nearest enlarge an image? integer) shrink an image? ∆ If our samples are apart, we can write this as: Before we try these operations, it’s helpful to think about images in a more mathematical way… f[i ,j] = Quantize{ f(i ∆, j ∆) } f[i,j] j i 5 6 Fourier transforms 1D Fourier examples We can represent functions as a weighted sum of Spatial domain Frequency domain sines and cosines. cos(2πax) We can think of a function in two complementary x s ways: -a a Spatially in the spatial domain π Spectrally in the frequency domain cos(2 bx) The Fourier transform and its inverse convert x s -b b between these two domains: II (x) sinc(s) x s ∞ Fs()= f ( xe )−isx2π dx ³ σ σ Spatial −∞ Frequency gauss(x; ) gauss(s; 1/ ) domain domain ∞ x s π fx()= ³ Fse ()isx2 ds −∞ gauss(x; 1/σ) gauss(s; σ) x s 7 8 2D Fourier transform Convolution One of the most common methods for filtering a ∞∞ −+π function is called convolution. Fs(,) s= f (,) xyeisxsy2(xy ) dxdy xy ³³ Spatial −∞ −∞ Frequency ∞∞ In 1D, convolution is defined as: domain π + domain fxy(,)= Fs ( , s ) eisxsy2(xy ) dsds ³³ xy x y −∞ −∞ gx()=∗ f () x hx () ∞ =−³ fxhx(')( xdx ')' Spatial domain Frequency domain −∞ ∞ =−³ fxhx(')(' xdx ) ' −∞ where hx( ) =− h( x ). 9 10 Convolution in 2D Convolution theorems In two dimensions, convolution becomes: Convolution theorem: Convolution in the spatial domain is equivalent to multiplication in the gxy(,)=∗ f (,) xy hxy (,) frequency domain. ∞∞ =−−³³f( x ', y ') h ( x x ', y y ') dx ' dy ' fh∗←→⋅ FH −∞ −∞ ∞∞ =−−³³fxyhxxyydxdy(',')(',')'' −∞ −∞ Symmetric theorem: Convolution in the frequency domain is equivalent to multiplication in the spatial where hxy( , ) =−− h( x , y ). domain. fh⋅←→∗ F H * = f(x,y) h(x,y) g(x,y) 11 12 1D convolution theorem example 2D convolution theorem example Spatial domain Frequency domain F(s ,s ) f(x) F(s) f(x,y) x y x s * * h(x) H(s) h(x,y) H(s ,s ) x s x y = = g(x) G(s) x s g(x,y) G(sx,sy) 13 14 The delta function Sifting and shifting The Dirac delta function, δ(x), is a handy tool for For sampling, the delta function has two important sampling theory. properties. It has zero width, infinite height, and unit area. Sifting: δδ−= − It is usually drawn as: fx()( x a ) fa ()( x a ) f(x) δ(x-a) = f(a)δ(x-a) x x x δ(x) a a a x Shifting: fx()∗−=−δ ( x a ) fx ( a ) f(x) δ f(x-a) * (x-a) = x x x a a 15 16 The shah/comb function Sampling A string of delta functions is the key to sampling. The Now, we can talk about sampling. resulting function is called the shah or comb function: ∞ III()x =−¦ δ (xnT ) n=−∞ x s which looks like: III(x) III(x) III(s) x T x s T so Amazingly, the Fourier transform of the shah = function takes the same form: = ∞ III()ssns=−δ ( ) ¦ o x s n=−∞ where so = 1/T. III(s) The Fourier spectrum gets replicated by spatial s sampling! so How do we recover the signal? 17 18 Sampling and reconstruction Sampling and reconstruction in 2D x s * x s = = x s Reconstruction x filter s = = x s 19 20 Sampling theorem Reconstruction filters This result is known as the Sampling Theorem and is The sinc filter, while “ideal”, has two drawbacks: due to Claude Shannon who first discovered it in 1949: It has large support (slow to compute) It introduces ringing in practice A signal can be reconstructed from its samples without loss of information, if the original signal We can choose from many other filters… has no frequencies above ½ the sampling frequency. sinc(x) For a given bandlimited function, the minimum rate x = x at which it must be sampled is the Nyquist frequency. II(x) = x x x * Λ (x) = x x gauss(x) x = x 21 22 Reconstruction filters in 2D Aliasing We can also perform reconstruction in 2D… What if we go below the Nyquist frequency? x s sinc(x) sinc(y) x * y x s II(x) II(y) x = = = Aliasing y x * x s Λ(x) Λ(y) x = y * y x s gauss(x) gauss(y) = x = = x s 23 24 Anti-aliasing Anti-aliasing by prefiltering Anti-aliasing is the process of removing the We can fill the “magic” box with analytic pre-filtering frequencies before they alias. of the signal: x s x s Magic * x s x s = = = = x s * x s x s = = Why may this not generally be possible? x s 25 26 Filtered downsampling Alternatively, we can sample the image at a higher rate, and then filter that signal: x s * x s = = x s We can now sample the signal at a lower rate. The whole process is called filtered downsampling or supersampling and averaging down. 27.