Computational Photography Filtering: Smoothing Objectives Example
Total Page:16
File Type:pdf, Size:1020Kb
Filtering: Smoothing • Point-process and Neighboring Pixels Computations on an Image for Image Smoothing Computational Photography CS 4475/6475 Maria Hybinette 1 Maria Hybinette 2 Maria Hybinette Objectives From Pixel/Point Operations to Groups of Pixels • Smooth an image while considering a neighborhood n of pixels – Average filtering (use the average of n) – Median filtering (use the median of n) • special non-linear filtering and smoothing • Previously we used pixel to pixel arithmetic (e.g., add(), sub())) approach • How to Smooth a Signal? • Approach: Use a group of pixels : Neighboring Values of a neighborhood. 1. Moving Average [1 1 1 1 1] X 1/5 2. Weighted Moving Average [1 4 6 4 1] X 1/16 3 Maria Hybinette 4 Maria Hybinette Example: 3x3 Kernel Example: 3x3 Kernel • Smoothing Process over an Image using Averages • 1*90/9=10, 2*90/9=20, …. 5 Maria Hybinette 6 Maria Hybinette Results in Shades of Gray • What about the edges? 7 Maria Hybinette 8 Maria Hybinette Edges Mathematical Notation, Observations, and Computing the Result Matrix • A Small image ‘h’ is “rubbed” over (larger) image ‘F’ that we are smoothing • Kernel: h – Example: 3x3 area around each original pixel used – Neighborhood size – k=1, window size = 3. – k >1, • Window size is 2k+1 – k=1, 2(1)+1 = 3 – Our kernel is 3x3 – Recall : A pixel is referenced by (i, j) • Expand the image we are smoothing: • Want Result Matrix ‘G’ – Wrap around pixels. (3x3 filter needs one more pixel – F à G using h. on each border, 5x5 needs 2) – G = h (operation) F – 9 Copy edges Maria Hybinette 10 Maria Hybinette Computing the Values ofA Result Mathematical Matrix RepresentationSmoothing when for a- iSmoothing are all 1s. Observations (continued)1 G[3, 3] = (A + B + C + D + E + FG[3, 3] = 1/9 (A + B + C + D + E + F + G + H + I ) + G + H + I) a b c 20 20 10 20 10 20 10 10 13 9 30 0 0 0 0 0 0 0 30 1 k k d e f k k G[i, j] = F[i + u, j + v] Let’s refer to F[i,j] as input and 1 2 ∑ ∑ 20 0 90A 90B 90C 90 90 0 20 20 20 10 20 10 20 10 10 13(2k +1) u k v k • Let’s refer to G[i, j]= F [i + u, j + v] =− =− g h i 20 0 90 0 90 90 90 0 20 2 30 0 0 0 0 0 0 0 30 20 0 D E F 90 90 0 20 G[i,j] as output, h[i,j] as the kernel(2k + 1) 1 – F[i,j] as input and u= k v=20 0k 90A 90B 90C 90 90 0 20 10 0 90 90 90 90 90 0 10 X− X− • ( 2 k + 1 ) 2 “Standardizes” the weight to 10 0 G H I 90 90 0 10 – G[i,j] as output 20 0 90D E0 90F 90 90 0 20 10 0 90 90 90 90 90 0 10 a b c A B C 10 0 90G 90H 90I 90 each90 0 10 pixel . (k=1, then 1/9) 10 0 90 90 90 90 90 0 10 G[3, 3] = a A + b B + c C + d D 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 k90 k 0 10 10 0 90 90 90 90 90 0 10 – h[i,j] as the kernel⇤ ⇤ ⇤ ⇤ d e f D E F F[i u, j v] + e E + f F + h H + i I A 10Mathematical0 90 90 90•90 ∑ 90 ∑ 0 10 + + Loop over allRepresentation pixels in 20 0 0 0 0 0 0 for0 20 Smoothing u=−k v=−k 20 0 0 0 0 0 0 0 20 • Example: pixel⇤ (3,3) ⇤(expanded⇤ edges⇤ above)g h i G H I 20 0 0 0 0 0 neighborhood0 0 20 around particular 20 20 10 20 10 20 10 10 13 20 20 10 20 10 20 10 10 13 20 20 10 20 10 20 10 10 13 – G[3, 3] = a*A + b*B + c*C + d*D +e*E + f*F + h*H + i*I 1 image pixel F[i,j] Attribute uniform1/9 1/9 1/9 Loop over all pixels in a b c 20 20 10 20 10 20 10 10 13 1 G[3, 3] = (A +– Bstarting+ C from+ upperD + rightE corner,+ F -k,+ -k,G if+ H + I) 1/9 1/9 1/9 9 G[3, 3] = (A + B + C + D + E + F + G + H + I) k=1, then 1 left, from center and up 1. 30 0 0 0 0 0 0 0 30 9 k k d e f – If all weights (a through i) are 1: weight to1/9 each1/9 1/9 neighborhood1 around 20 0 90A 90B 90C 90 90 0 20 G[i, j]= F [i + u, j + v] g h i 11 – G[3, 3] = 1/9 (A + B + C + D + E + F + G + H + I ) Maria Hybinette (2k 12+ 1)2 Maria Hybinette 20 0 90D 0E 90F 90 90 0 20 © 2015 Irfan Essa, Georgia Tech, All Rights Reserved pixel imageu= k v =pixelk F[i,j] 10 0 90 90 90 90 90 0 10 X− X− 10 0 G H I 90 90 0 10 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 90 0 10 20 0 0 0 0 0 0 0 20 20 0 0 0 0 0 0 0 20 © 2015 Irfan Essa, Georgia Tech, All Rights Reserved 20 20 10 20 10 20 10 10 13 Attribute uniform Loop over all pixels in20 20 10 20 10 20 10 10 13 weight to each neighborhood around pixel image pixel F[i,j] © 2015 Irfan Essa, Georgia Tech, All Rights Reserved A Mathematical RepresentationGeneralize the foruniform Smoothing weight Example: Box Filter for Smoothing 1 G[3, 3] = (A + B + C + D + E +G[3, 3] = 1/9 (A + B + C + D + E + F + G + H + I ) F + G + H + I) a b c 20 20 10 20 10 20 10 10 13 9 k k 30 0 0 0 0 0 0 0 30 k k 1 d e f G[i, j] = F[i + u, j + v] 20 0 90A 90B 90C 90 90 0 20 1 (2k +1)2 ∑ ∑ G[i, j]= F [i + u, j + v] u=−k v=−k g h i (2k + 1)2 20 0 90D 0E 90F 90 90 0 20 u= k v= k 10 0 90 90 90 90 90 0 10 X− X− • More Generally, 10 0 G H I 90 90 0 10 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 90 0 10 G[i, j] = a * A + b* B + c*C + d * D + e* E + f * F + h * H +i* I 20 0 0 0 0 0 0 0 20 A Mathematical Representation20 0 0 0 0 0 0 0for20 Smoothing k k 20 20 10 20 10 20 10 10 13 G[i, j] = ∑ ∑ h[u, v]* F[i + u, j + v] 20 20 10 20 10 20 10 10 13 1 u=−k v=−k • Uniform Kernel (all 1s). Attribute uniformG[3, 3] = Loop( A + B over+ C + Dall+ E +pixelsF + G + inH + I) a b c 20 20 10 20 10 20 10– Box10 Area13 above is 21x21 the larger the kernel the ‘smoother’ the 9 result. • Formula is Referred to Cross 30 0 0 0 0 0 0 0 30 k k d e f • The larger the box the smoother it gets. weight to each neighborhood1Correlation more onaround this later. 20 0 90A 90B 90C 90 90 0 20 G[i, j]= F [i + u, j + v] g h i (2k 13+ 1)2 Maria Hybinette 20 0 90D 0E 90F14 90 h7ps://90 0 en.wikipedia.org20 /wiki/Box_blur Maria Hybinette pixel imageu= k v =pixelk F[i,j] 10 0 90 90 90 90 90 0 10 X− X− 10 0 G H I 90 90 0 10 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 90 0 10 10 0 90 90 90 90 90 0 10 20 0 0 0 0 0 0 0 20 20 0 0 0 0 0 0 0 20 © 2015 Irfan Essa, Georgia Tech, All Rights Reserved 20 20 10 20 10 20 10 10 13 20 20 10 20 10 20 10 10 13 SpecialSpecialAttribute Case:Case: uniform MedianMedian Loop over Filtering all pixels in Specialweight Example:to eachCase: Median neighborhoodMedian Filter for smoothing aroundFiltering Comparison 20 20 1 20 10 20 10 13 13 Special Case: Median Filtering 20 20 1 20 10 20 10 13 13 pixel 20 20 1 image pixel F[i,j] 0 0 200 200 120 2020 10 20 10 13 13 20 20 1 20 0 0 0 0 0 0 20 20 20 20 1 20 10 20 10 13 13 20 0 0 0 0 0 0 20 20 20 0 0 20 2020 20 1 median(20 0 0 0 0 0 0 20)20 median( 0 0 0 1 20 20 20 20 90 ) 90 902090 090 020 020 0 0 0 20 20 20 0 0 20 0 0 0 0 0 1 20 20 20 20 90 = 20 20 0 90 90 90 90 90 20 20 median(20 0 90 90 90 90 90 20)20 median(median( ) median( 0 0 0 1 20 20 ) 20= 20 90 ) 20 0 90 90 90 90 90 20 20 20 0 90 20 200 0 900 0 0 0 1 20=20 2020 20 90 20 0 90 90 90 90 90 20 2020 02090median(0 090 90 90 20 20 ) Medianmedian( = 20 (reMoves outliers) ) = 20 20 0 90 0 90 90 90 20 20 10 0 90 90 90 90 90 10 10 20 0 90 0 0 0 1 20 20 20 20 90 20 0 90 0 90 90 90 20 20 average( ) = 19 20 0 90 0 90 90 90 20 2010 0 90 90 90 90 90 10 10 10 0 90 90 90 90 ©90 2015 Irfan10 Essa,10 Georgia Tech, All Rights Reserved 10 0 90 90 90 90 90 10 10 0 0 0 1 20 20 20 20 90 10 0 90 90 90 90 90 10 10 average( 0 0 0 1 20 20 20 20 90 ) = 19 10 0 90 90 90 90 90 10 1020 0 0 0 0 0 0 20 20 average(F [i,j] 0 0 0 )1 20= 20 1920 20 90 10 0 90 90 90 90 90 10 10 20 20 10 20 10 20 10 13 13 average( ) = 19 10 0 90 90 90 90 90 10 10 Average=171/9 = 19 10 0 90 90 901090 090 9010 9010 90 90 90 10 10 10 0 90 90 90 90 90 10 10 Apply median filter to all 3x3 regions over the 20 0 0 0 0 100 00 9020 9020 90 90 90 10 10 20 0 0 0 0 0 0 20 20 FF [i,j][i,j] • Medianwhole Filterimage :(as Nonlinear we would filter for averaging) 20 20 10 20 102020 010 013 013 0 0 0 20 20 F [i,j] 20 20 10 20 10 20 10 13 13 © 2015 Irfan Essa,– Georgia Reduces Tech, All Rights Reserved noise (the outliers are ‘basically’ noise and 20 20 10 20 10 20 10 13 13 removed).