Lecture 8 Image Transformations
Total Page:16
File Type:pdf, Size:1020Kb
Lecture 8 Image Transformations Last Time (global and local warps) Handouts: PS#2 assigned Idea #1: Cross-Dissolving / Cross-fading Idea #2: Align, then cross-disolve Interpolate whole images: Ihalfway = α*I1 + (1- α)*I2 This is called cross-dissolving in film industry Align first, then cross-dissolve Alignment using global warp – picture still valid But what if the images are not aligned? Failure: Global warping Idea #3: Local warp & cross-dissolve Warp Warp What to do? Cross-dissolve doesn’t work Avg. ShapeCross-dissolve Avg. Shape Global alignment doesn’t work Cannot be done with a global transformation (e.g. affine) Morphing procedure: Any ideas? 1. Find the average shape (the “mean dog”☺) Feature matching! local warping Nose to nose, tail to tail, etc. 2. Find the average color This is a local (non-parametric) warp Cross-dissolve the warped images 1 Triangular Mesh Transformations 1. Input correspondences at key feature points 2. Define a triangular mesh over the points (ex. Delaunay Triangulation) (global and local warps) Same mesh in both images! Now we have triangle-to-triangle correspondences 3. Warp each triangle separately How do we warp a triangle? 3 points = affine warp! Just like texture mapping Slide Alyosha Efros Parametric (global) warping Parametric (global) warping Examples of parametric warps: T p’ = (x’,y’) p = (x,y) Transformation T is a coordinate-changing machine: p’ = T(p) translation rotation aspect What does it mean that T is global? Is the same for any point p can be described by just a few numbers (parameters) Let’s represent T as a matrix: p’ = Tp affine perspective cylindrical x' x = T y' y Scaling Scaling Scaling a coordinate means multiplying each of its Non-uniform scaling: different scalars per component: components by a scalar Uniform scaling means this scalar is the same for all components: x * 2, y *0.5 * 2 2 Scaling 2-D Rotation Scaling operation: x'= ax y'= by (x’, y’) Or, in matrix form: x' a 0x (x, y) = y' 0 by x’ = x cos(θ) - y sin(θ) scaling matrix S θ y’ = x sin(θ) + y cos(θ) What’s inverse of S? 2-D Rotation 2-D Rotation This is easy to capture in matrix form: x = r cos (φ) y = r sin (φ) x’ = x cos(θ) - y sin(θ) x' cos()θ − sin(θ )x = (x’, y’) x’ = r cos (φ + θ) y’ = x sin(θ) + y cos(θ) y' sin()θ cos ()θ y y’ = r sin (φ + θ) (x, y) R Trig Identity… x’ = r (cos(φ) cos(θ) – sin(φ) sin(θ)) Even though sin(θ) and cos(θ) are nonlinear functions θ y’ = r (sin(φ) cos(θ) + cos(φ) sin(θ)) of θ, φ x’ is a linear combination of x and y Substitute… y’ is a linear combination of x and y x’ = x cos(θ) - y sin(θ) What is the inverse transformation? y’ = x sin(θ) + y cos(θ) Rotation by – θ −1 T For rotation matrices R = R 2x2 Matrices 2x2 Matrices What types of transformations can be What types of transformations can be represented with a 2x2 matrix? represented with a 2x2 matrix? 2D Identity? 2D Rotate around (0,0)? x'= x x' 1 0x = x'= cosΘ* x − sin Θ* y x' cosΘ − sin Θx y'= y y' 0 1y = y'= sin Θ* x + cosΘ* y y' sin Θ cosΘ y 2D Scale around (0,0)? 2D Shear? x'= x + shx * y x' 1 sh x x'= s x * x x' s 0 x x x = = y'= shy * x + y y' s * y y' shy 1 y = y y' 0 s y y 3 2x2 Matrices 2x2 Matrices What types of transformations can be What types of transformations can be represented with a 2x2 matrix? represented with a 2x2 matrix? 2D Mirror about Y axis? 2D Translation? x' −1 0 x x'= x + t x'= −x = x y'= y y' 0 1y NO! y'= y + t y 2D Mirror over (0,0)? Only linear 2D transformations can be represented with a 2x2 matrix x'= −x x' = −1 0 x y'= −y y' 0 −1y All 2D Linear Transformations Homogeneous Coordinates Q: How can we represent translation as a Linear transformations are combinations of … 3x3 matrix? Scale, Rotation, x' a bx = x'= x + t Shear, and y' c d y x Mirror y'= y + t Properties of linear transformations: y Origin maps to origin Lines map to lines Parallel lines remain parallel Ratios are preserved Closed under composition e f i j x' = a b x y' c d g h k l y Homogeneous Coordinates Homogeneous Coordinates Homogeneous coordinates Add a 3rd coordinate to every 2D point represent coordinates in 2 dimensions with a 3-vector (x, y, w) represents a point at location (x/w, y/w) (x, y, 0) represents a point at infinity (0, 0, 0) is not allowed x x y homogeneou s coords 2 → y (2,1,1) or (4,2,2) or (6,3,3) y 1 1 Convenient 1 2 x coordinate system to represent many useful transformations 4 Homogeneous Coordinates Translation Q: How can we represent translation as a Example of translation Homogeneous Coordinates 3x3 matrix? x'= x + t x x' 1 0 t x x x + t x y'= y + t y y' = 0 1 t y = y + t y y 1 0 0 1 1 1 A: Using the rightmost column: 1 0 t x Translation = 0 1 t y tx = 2 0 0 1 t = 1 y Basic 2D Transformations Matrix Composition Basic 2D transformations as 3x3 matrices Transformations can be combined by matrix multiplication x' 1 0 tx x x' sx 0 0x y' = 0 1 t y y' = 0 s 0 y y y x' 1 0 tx cosΘ − sin Θ 0 sx 0 0 x 1 0 0 11 1 0 0 11 y' = 0 1 tysin Θ cosΘ 0 0 sy 0 y Translate Scale w' 0 0 1 0 0 1 0 0 1w p’ = T(tx,ty) R(Θ) S(sx,sy) p x' cosΘ − sin Θ 0x x' 1 shx 0x y' = sin Θ cosΘ 0y y' = sh 1 0 y y 1 0 0 11 1 0 0 11 Rotate Shear Affine Transformations Projective Transformations Affine transformations are combinations of x' a b c x y' d e f y x' a b c x Linear transformations, and = Projective transformations … w 0 0 1 w y' = d e f y Translations Affine transformations, and w' g h i w Properties of affine transformations: Projective warps Origin does not necessarily map to origin Properties of projective transformations: Lines map to lines Origin does not necessarily map to origin Parallel lines remain parallel Lines map to lines Ratios are preserved Parallel lines do not necessarily remain parallel Closed under composition Models change of basis Ratios are not preserved Closed under composition Will the last coordinate w always be 1? Models change of basis 5 2D image transformations Recovering Transformations ? T(x,y) yy xx f(x,y) g(x’,y’) What if we know f and g and want to recover the transform T? willing to let user provide correspondences How many do we need? x′ a b c x y′ = d e f y These transformations are a nested set of groups • Closed under composition and inverse is a member 1 0 0 11 Translation: # correspondences? Euclidian: # correspondences? ? ? T(x,y) T(x,y) yy yy xx xx How many correspondences needed for translation? How many correspondences needed for translation + How many Degrees of Freedom? rotation? What is the transformation matrix? How many DOF? 1 0 p' − p x x cosθ − sinθ tx T = 0 1 p' − p y y T = ssinθ cosθ t y 0 0 1 0 0 1 Affine: # correspondences? Projective: # correspondences? p 2 q 2 p 2 p4 ? q 2 q4 T(x,y) ? q yy3 q T(x,y) p 1 xx3 yyq1 q3 p3 How many correspondences needed for affine? xx How many DOF? An affine transformation is a composition of translations, rotations, How many correspondences needed for projective? dilations, and shears. How many DOF? 1 0 tx cosθ − sinθ 0 sx 0 0 1 kx 0 sinθ cosθ 0 0 1 t y 0 sy 0 k y 1 0 0 0 1 0 0 1 0 0 1 0 0 1 translationsrotation dilations shear 6 Warping triangles Hint: Warping triangles q p2 2 (Barrycentric Coordinates) (0,1) ? T(p) q3 p1 p 3 q1 (0,0) (1,0) Source Destination −1 T T2 p2 1 q2 Given two triangles: p1p2p3 and q1’q2’q3’ in 2D (6 constraints) change Need to find transform T to transfer all pixels from one to the other. Inverse change of basis What kind of transformation is T? of basis affine q3 How can we compute the transformation matrix: p1 p3 Sourceq1 Destination x' a b c x y' = d e f y 1 0 0 1 1 Trick: Computing T Warping sequence q2 (0,1) p (1-α) 2 V α q q3 (0,0) (1,0) q p 1 Destination T2 p1 p q2 3 Source Project and Event Manager change of basis v = p + α(t)()q − p q3 T2 = []q2 − q1 q3 − q1 q1 q1 Destination = ()1−α(t) p + α(t) q v = ()1−α(t) p + α(t) Tp where α is a function of time Forward warping Forward warping T(x,y) yy’ xx’ T(x,y) f(x,y) g(x’,y’) yy’ xx’ f(x,y) g(x’,y’) Send each pixel f(x,y) to its corresponding location Send each pixel f(x,y) to its corresponding location (x’,y’) = T(x,y) in the second image (x’,y’) = T(x,y) in the second image Q: what if pixel lands between four pixels? Q: what if pixel lands between four pixels? A: distribute color among neighboring pixels (x’,y’) – Known as “splatting” – Check out griddata in Matlab 7 Inverse warping Inverse warping T-1(x,y) y y’ T-1(x,y) xxx’ y y’ f(x,y) g(x’,y’) xxx’ f(x,y) g(x’,y’) Get each pixel g(x’,y’) from its corresponding Get each pixel g(x’,y’) from its corresponding location location (x,y)=T-1(x’,y’) in the first image (x,y)=T-1(x’,y’) in the first image Q: what if pixel comes from between four pixels? Q: what if pixel comes from between four pixels? A: Interpolate color value from neighbors – nearest neighbor, bilinear, Gaussian, bicubic – Check out interp2 in Matlab Forward vs.