CS664 Computer Vision 5. Image Geometry Dan Huttenlocher First Assignment Wells paper handout for question 1 Question 2 more open ended – Less accurate approximations • Simple box filtering doesn’t work – Anisotropic, spatially dependent 2 Image Warping Image filtering: change range of image g(x) = T¸f(x) f g T Image warping: change domain of image g(x) = f(T(x)) f g T 3 Feature Detection in Images Filtering to provide area of support – Gaussian, bilateral, … Measures of local image difference – Edges, corners More sophisticated features are invariant to certain transformations or warps of the image – E.g., as occur when viewing direction changes 4 Parametric (Global) Warping Examples of parametric warps: translation rotation projective affine cylindrical 5 Parametric (Global) Warping T p = (x,y) p’ = (x’,y’) p’ = T(p) What does it mean that T is global? – Same function for any point p – Described by a few parameters, often matrix x' x p’ = M*p ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ = M⎢ ⎥ ⎣y'⎦ ⎣y⎦ 6 Scaling Scaling a coordinate means multiplying each of its components (axes) by a scalar Uniform scaling means this scalar is the same for all components: × 2 7 Scaling Non-uniform scaling: different scalars per component: X × 2, Y × 0.5 8 Scaling Scaling operation: x'= ax y'= by Or, in matrix form: ⎡x'⎤ ⎡a 0⎤⎡x⎤ ⎢ ⎥ = ⎢ ⎥⎢ ⎥ ⎣y'⎦ ⎣0 b⎦⎣y⎦ scaling matrix S What’s inverse of S? 9 2-D Rotation (x’, y’) About origin (x, y) x’ = x cos(θ) - y sin(θ) θ y’ = x sin(θ) + y cos(θ) 10 2-D Rotation This is easy to capture in matrix form: ⎡x'⎤ ⎡cos(θ ) − sin(θ )⎤⎡x⎤ ⎢ ⎥ = ⎢ ⎥⎢ ⎥ ⎣y'⎦ ⎣sin()θ cos ()θ ⎦⎣y⎦ R Even though sin(θ) and cos(θ) are nonlinear functions of θ, – x’ is a linear combination of x and y – y’ is a linear combination of x and y Inverse transformation, rotation by –θ – For rotation matrices, det(R) = 1 and R −1 = R T 11 2x2 Transformation Matrices What types of transformations can be represented with a 2x2 matrix? 2D Identity? (A rotation) x'= x ⎡x'⎤ = ⎡1 0⎤⎡x⎤ y'= y ⎣⎢y'⎦⎥ ⎣⎢0 1⎦⎥⎣⎢y⎦⎥ 2D Scale around (0,0)? x'= s x * x ⎡x'⎤ ⎡sx 0 ⎤⎡x⎤ ⎢ ⎥ = ⎢ ⎥⎢ ⎥ y'= s y * y ⎣ y'⎦ ⎣ 0 s y ⎦⎣ y⎦ 12 2x2 Transformation Matrices What types of transformations can be represented with a 2x2 matrix? 2D Rotate around (0,0)? ⎡x'⎤ ⎡cosΘ − sin Θ⎤⎡x⎤ x'= cosΘ* x − sin Θ* y ⎢ ⎥ = ⎢ ⎥⎢ ⎥ y'= sin Θ* x + cosΘ* y ⎣ y'⎦ ⎣sin Θ cosΘ ⎦⎣ y⎦ 2D Shear? ⎡x'⎤ ⎡ 1 sh ⎤⎡x⎤ = x x'= x + shx * y ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎣ y'⎦ ⎣shy 1 ⎦⎣ y⎦ y'= shy * x + y 13 2x2 Transformation Matrices What types of transformations can be represented with a 2x2 matrix? 2D Mirror about Y axis? x'= −x ⎡x'⎤ = ⎡−1 0⎤⎡x⎤ y'= y ⎣⎢y'⎦⎥ ⎣⎢ 0 1⎦⎥⎣⎢y⎦⎥ 2D Mirror over (0,0)? x'= −x ⎡x'⎤ = ⎡−1 0 ⎤⎡x⎤ y'= −y ⎣⎢y'⎦⎥ ⎣⎢ 0 −1⎦⎥⎣⎢y⎦⎥ 14 All 2D Linear Transformations Linear transformations are combinations of … – Scale, ⎡x'⎤ ⎡a b⎤⎡x⎤ – Rotation, = – Shear, and ⎢y'⎥ ⎢c d⎥⎢y⎥ – Mirror ⎣ ⎦ ⎣ ⎦⎣ ⎦ Properties of linear transformations: – Origin maps to origin – Lines map to lines – Parallel lines remain parallel – Ratios are preserved – Closed under composition e f i j ⎡x'⎤ = ⎡a b⎤⎡ ⎤⎡ ⎤⎡x⎤ ⎣⎢y'⎦⎥ ⎣⎢c d⎦⎥⎣⎢g h⎦⎥⎣⎢k l ⎦⎥⎣⎢y⎦⎥ 15 Linear Transformations as Change of Basis j =(0,1) v =(vx,vy) p’ p u=(ux,uy) i =(1,0) p ’=4u +3v p=4i+3j = (4,3) p’=4u+3v x x x py’=4uy+3vy ⎡u x v x ⎤⎡4⎤ ⎡u x v x ⎤ p' = ⎢ ⎥⎢ ⎥ = ⎢ ⎥p ⎣u y v y ⎦⎣3⎦ ⎣u y v y ⎦ Any linear transformation is a basis! 16 2x2 Transformation Matrices What types of transformations can be represented with a 2x2 matrix? 2D Translation? x'= x + t x NO! y'= y + t y Only linear 2D transformations can be represented with a 2x2 matrix 17 Homogeneous Coordinates How can we represent translation as matrix? x'= x + t x y'= y + t y Homogeneous coordinates – Represent coordinates in 2 dimensions with a 3- vector ⎡x⎤ x ⎡ ⎤ homogeneous coords ⎢ ⎥ ⎢ ⎥ ⎯⎯→⎯⎯⎯ ⎢y⎥ ⎣y⎦ ⎣⎢1⎦⎥ 18 Homogeneous Coordinates Add a 3rd coordinate to every 2D point – (x, y, w) represents a point at 2D location (x/w, y/w) – (x, y, 0) represents a point at infinity – (0, 0, 0) is not allowed y 2 (2,1,1) or (4,2,2) or (6,3,3) 1 1 2 x 19 Homogeneous Coordinates How can we represent translation as matrix? x'= x + t x y'= y + t y Last column of homogeneous matrix ⎡1 0 t x ⎤ ⎢ ⎥ Translation = ⎢0 1 t y ⎥ ⎢ ⎥ ⎣0 0 1 ⎦ 20 Translation Example of translation in homogeneous coordinates ⎡ x'⎤ ⎡1 0 t x ⎤⎡ x ⎤ ⎡ x + t x ⎤ ⎢ ⎥ ⎢ y'⎥ = 0 1 t ⎢ y⎥ = ⎢ y + t ⎥ ⎢ ⎥ ⎢ y ⎥⎢ ⎥ ⎢ y ⎥ ⎢ ⎥ ⎣⎢ 1 ⎦⎥ ⎣0 0 1 ⎦⎣⎢1 ⎦⎥ ⎣⎢ 1 ⎦⎥ tx = 2 ty = 1 21 Homogeneous 2D Transformations Basic 2D transformations as 3x3 matrices ⎡x'⎤ ⎡1 0 tx ⎤⎡x⎤ ⎡x'⎤ ⎡sx 0 0⎤⎡x⎤ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢y'⎥ = ⎢0 1 t ⎥⎢y⎥ y' = 0 s 0 y ⎢ ⎥ ⎢ y ⎥⎢ ⎥ ⎢ ⎥ ⎢ y ⎥⎢ ⎥ ⎣⎢ 1 ⎦⎥ ⎣⎢0 0 1 ⎦⎥⎣⎢1⎦⎥ ⎣⎢ 1 ⎦⎥ ⎣⎢ 0 0 1⎦⎥⎣⎢1⎦⎥ Translate Scale ⎡x'⎤ ⎡cosΘ − sin Θ 0⎤⎡x⎤ ⎡x'⎤ ⎡ 1 shx 0⎤⎡x⎤ ⎢ ⎥ ⎢ ⎥⎢ ⎥ y' = sin Θ cosΘ 0 y ⎢ y'⎥ = ⎢sh 1 0⎥⎢ y⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ y ⎥⎢ ⎥ ⎣⎢1 ⎦⎥ ⎣⎢ 0 0 1⎦⎥⎣⎢1⎦⎥ ⎣⎢ 1 ⎦⎥ ⎣⎢ 0 0 1⎦⎥⎣⎢1⎦⎥ Rotate Shear 22 Affine Transformations Affine transformations are … ⎡x'⎤ ⎡a b c ⎤⎡ x⎤ ⎢y'⎥ = ⎢d e f ⎥⎢ y⎥ – Linear transformations, and ⎢w⎥ ⎢0 0 1 ⎥⎢w⎥ – Translations ⎣ ⎦ ⎣ ⎦⎣ ⎦ Properties of affine transformations: – Origin does not necessarily map to origin – Lines map to lines – Parallel lines remain parallel – Ratios are preserved – Closed under composition – Models change of basis – Maps any triangle to any triangle (or parallelogram) 23 Projective Transformations ⎡ ⎤ ⎡ ⎤⎡ ⎤ Projective transformations … x' a b c x ⎢ y'⎥ = ⎢d e f ⎥⎢ y⎥ – Affine transformations, and ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎣w'⎦ ⎣g h i ⎦⎣w⎦ – Projective warps Properties of projective transformations: – Origin does not necessarily map to origin – Lines map to lines – Parallel lines do not necessarily remain parallel – Ratios are not preserved – Closed under composition – Models change of basis – Maps any quadrilateral to any quadrilateral 24 Matrix Composition Transformations can be combined (composed) by matrix multiplication ⎡ x'⎤ ⎛ ⎡1 0 tx⎤ cosΘ − sin Θ 0 ⎡sx 0 0⎤⎞⎡ x⎤ ⎜ ⎡ ⎤ ⎟ ⎢ y'⎥ = ⎢0 1 ty⎥⎢sin Θ cosΘ 0⎥⎢ 0 sy 0⎥ ⎢ y⎥ ⎢ ⎥ ⎜ ⎢ ⎥ ⎢ ⎥⎟⎢ ⎥ ⎣w'⎦ ⎝ ⎣0 0 1 ⎦⎣⎢ 0 0 1⎦⎥⎣ 0 0 1⎦⎠⎣w⎦ p’ = T(tx,ty) R(Θ) S(sx,sy) p 25 2D image transformations • Nested set of groups 26 Affine Invariants Affine transformations preserve collinearity of points: – If 3 points lie on the same line, their images under affine transformations also line on the same line and, – The middle point remains between the other two points Concurrent lines remain concurrent (images of intersecting lines intersect), Ratio of length of line segments of a given line remains constant Ratio of areas of two triangles remains constant, Ellipses remain ellipses and the same is true for parabolas and hyperbolas 27 3D Interpretations Orthographic projection plus scaling of Euclidean motion in 3D world yields 2D affine transformation – Often called weak perspective imaging model Set of coplanar points in 3D – Undergo rigid body motion – Scaling – analogous to perspective size with distance but same scaling for all the points – Projection into image plane (drop coordinate) 28 Orthographic Projection Image World Also called “parallel projection”: (x, y, z) → (x, y) What’s the projection matrix (homogeneous coords)? 29 Weak Perspective Imaging Scaling in addition to parallel projection Composed with 3D rigid body motion (6 dof) Equivalent to affine transformation of plane (6 dof) up to reflection – state without proof 30 Projection – Pinhole Camera Model Projection equations – Compute intersection with PP of ray from (x,y,z) to COP – Derived using similar triangles – Parallel projection where d infinite 31 Perspective Projection Homogeneous coordinates Divide by third coordinate Scaling projection matrix, no effect 32 Perspective Projection Composed with 3D rigid body motion (6 dof) Focal length, d, and distance z two additional parameters Equivalent to projective transformation of plane (homography) – 3x3 matrix in homogeneous coordinates, 8 dof – Again state without proof 2D affine and projective transformations correspond to images of plane in space under rigid motion, different imaging models 33 Image Warping T(x,y) yy’ xx’f(x,y) g(x’,y’) Given coordinate transform (x’,y’) = T(x,y) and source image f(x,y) How do we compute transformed image g(x’,y’) = f(T(x,y))? 34 Forward Warping T(x,y) yy’ xx’f(x,y) g(x’,y’) Send each pixel f(x,y) to its corresponding location (x’,y’) = T(x,y) in the second image Q: What about when pixel lands “between” two pixels? 35 Forward Warping Points at pixel T(x,y) centers yy’ xx’f(x,y) g(x’,y’) Send each pixel f(x,y) to its corresponding location (x’,y’) = T(x,y) in the second image Q: What about when pixel lands “between” two pixels? A: Distribute color among neighboring pixels (x’,y’) – Known as “splatting” 36 Inverse Warping T-1(x,y) y y’ xxx’f(x,y) g(x’,y’) Get each pixel g(x’,y’) from its corresponding location (x,y)=T-1(x’,y’) in the first image Q: What about when pixel comes from “between” two pixels? 37 Inverse Warping T-1(x,y) y y’ xxx’f(x,y) g(x’,y’) Get each pixel g(x’,y’) from its corresponding location (x,y)=T-1(x’,y’) in the first image Q: What about when pixel comes from “between” two pixels? A: Interpolate color value from neighbors – Nearest neighbor, bilinear, Gaussian, bicubic 38 Bilinear Interpolation (Reminder) Sampling at f(x,y): 39 Forward vs.