Lecture 09 – Projective Geometry
Total Page:16
File Type:pdf, Size:1020Kb
CITS 4402 Computer Vision Ajmal Mian Lecture 09 – Projective Geometry Overview of this lecture Vanishing points and lines Projective invariants Cross-ratio Measuring the height of objects from a single image Homography Image rectification CNN demo and discussion about the Project Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 2 Can you work out the geometry of this image? Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 3 Notice the pattern of the trees in this 1460 A.D. painting Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 4 Can you work out the geometry of this image? Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 5 Vanishing points In perspective image, all parallel lines meet at a single point. This point is called the “Vanishing Point”. Multiple sets of parallel lines will give multiple vanishing points. Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 6 Single vanishing point Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 7 Ancient painters were perhaps aware of this concept Carpaccio 1514 Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 8 Two vanishing points Can be outside the image Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 9 Vanishing points and vanishing line Joining two vanishing points give a vanishing line Vertical parallel lines give a vertical vanishing point Vertical vanishing point is at infinity here Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 10 Points at infinity What are the image coordinates of the point at infinity on the X-axis? 1 A point at infinity on the X-axis is represented by 0 . 0 0 Where will this appear in the image? 1 푠푢 푞11 푞12 푞13 푞14 푞11 0 푠푣 = 푞 푞 푞23 푞24 = 푞21 21 22 0 푠 푞 푞 푞33 푞34 푞 31 32 0 31 Similarly, points at infinity on the y-axis and z-axis are represented by the 2nd and 3rd columns. The projection of origin of the world coordinates is given by the last column. Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 11 Vanishing points and vanishing lines Notice how the parallel lines become non-parallel when projected on the image plane Vanishing points are often outside the image boundary Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 12 Vanishing points and vanishing lines Notice how the parallel lines become non-parallel when projected on the image plane Vanishing points are often outside the image boundary Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 13 The horizon Parallel lines that are not orthogonal to the optical axis meet at a vanishing point Two sets of parallel lines on the ground plane will give two vanishing points. Joining the vanishing points gives the vanishing line or “horizon” Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 14 Vanishing lines A set of parallel planes that are not parallel to the image plane intersect the image plane at a vanishing line. The horizon is a special vanishing line when the set of parallel planes are parallel to the ground reference. Anything in the scene that is above the camera will be projected above the horizon in the image. Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 15 How to calculate vanishing points and lines A point in image is defined by its (u,v) coordinates Two points determine a line Intersection of two parallel lines will give us a vanishing point Two vanishing points will give us the corresponding vanishing line Two clicks on each parallel line will solve the problem For automatic solution: Edge detection + Hough lines will solve the problem Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 16 Points and lines in Homogeneous coordinates 푥 Points in 2D are represented in homogeneous coordinates as a 3-vector 푦 . 1 Points and lines are dual in 2D,푎 so lines are represented in homogeneous coordinates as 3-vector also: 푏 . 푐 Homogeneous coordinates contain redundant information – the representation includes an arbitrary scale! e.g., 2 3 1 푇 and 4 6 2 푇 represent the same point 2 3 푇 in inhomogeneous coordinates. e.g., 4 7 −3 푇 and 2 3.5 −1.5 푇 represent the same line – as 4푥 + 7푦 − 3 = 0 and 2푥 + 3.5푦 − 1.5 = 0 denote the same line. To determine if a line 풍 = 푎 푏 푐 푇 passes through a point 풑 = 푥 푦 1 푇 check if their dot product is 0. That is, 풍푇풑 = 0 ⇒ 풍 passes through 풑. Example: Line 2 4 5 푇contains the point 1.5 −2 1 푇 since 2 4 5 ∙ 1.5 −2 1 = 2 ∗ 1.5 − 4 ∗ 2 + 5 ∗ 1 = 0. Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 17 Equation of a line given two points 푥1 푥2 The line 풍 that passes through two points 풑1 = 푦1 and 풑ퟐ = 푦2 is obtained via 1 1 the cross product 풍 = 풑1 × 풑2. Example: the line 풍 passing through the points 0 2 1 푇 and 3 0 1 푇 is: 0 3 2 2 × 0 = 3 . 1 1 −6 Thus, the equation of the line is 2푥 + 3푦 − 6 = 0. Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 18 Finding the intersection of two lines Since lines and points are dual in 2D, we obtain the line coordinates the same 푇 way as before. The point of intersection 풑 of two given lines 풍1 = 푎1 푏1 푐1 푇 and 풍2 = 푎2 푏2 푐2 is obtained via the cross product 풑 = 풍1 × 풍2. Example: The lines 4 6 2 푇 and 2 0 1 푇 intersection at 6 0 −12 푇 1 푇 which corresponds to the point − 0 in inhomogeneous coordinates. 2 Exercise: What are the coordinates of the intersection point of lines 3 1 2 푇 and 6 2 2 푇 ? Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 19 Can you find the equation of the vanishing line in this image? Click on red points to get the first line Click on the green points to get the second line Find their intersection 풑 Repeat the process for another set of parallel lines 풍 1 Cross produce of the two vanishing points will give the equation of the vanishing line 풍2 Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 20 What can we find from the vanishing points and lines? Given the horizon, the vertical vanishing point and one reference height. The height of any point (from the ground plane) can be computed by specifying the point and its projection on the ground plane in the image. Criminisi et al 1998. Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 21 Human height measurement - example If we know the height of one object in the scene We can find the height of a person using vanishing points Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 22 Representation of various transformations Euclidean transformation: 푟 푟 푟 푡 푟 푟 푡 11 12 13 1 11 12 1 푟 푡 2D: 푟 푟 푡 3D: 푟21 푟22 23 2 12 22 2 푟 푟 푟 푡 0 0 1 31 32 33 3 0 0 0 1 Similarity transformation: 푟 푟 푟 푡 푟 푟 푡 11 12 13 1 11 12 1 푟 푡 2D: 푟 푟 푡 3D: 푟21 푟22 23 2 12 22 2 푟 푟 푟 푡 0 0 푠 31 32 33 3 0 0 0 푠 Affine transformation: × × × × × × × × × × × 2D: × × × 3D: × × × × 0 0 1 0 0 0 1 Projective transformation: × × × × × × × × × × × 2D: × × × 3D: × × × × × × 1 × × × 1 Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 23 Geometric invariants Preserves angles Preserves Preserves ratios of lengths ratios of lengths concurrency ratios of areas ratios of areas collinearity parallelism parallelism cross-ratio concurrency concurrency collinearity collinearity cross-ratio cross-ratio Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 24 Cross-ratio Most important geometric invariant for projective transformation is the cross-ratio of 4 points on a line. The cross-ratio is the ratio of two ratios of lengths • Given 4 points on a line, say A, B, C and 퐷. st • Take 푥1 as the 1 reference point, compute 푟1 = 퐴퐶 퐴퐷. nd • Take 푥2 as the 2 reference point, compute 푟2 = 퐵퐶 퐵퐷. • Cross-ratio is defined as 푟 = 푟1 푟2. The two reference points and the ratios can also be arbitrary chosen e.g. 푟1 can be set to 퐶퐷 퐶퐵, 푟2 can be set to 퐴퐷 퐴퐵, and 푟 = 푟2 푟1. There are 4! = 24 permutations of possible cross-ratio values however, these values are repeated leaving only 6 unique cross-ratio possibilities Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 25 Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 26 Cross-ratio of equidistant points A B C D 1 1 1 Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 27 Measurements from an image… Let us measure it from an image with known equidistant points Cross-ratio can always be calculated from an image. This cross-ratio will be exactly the same as in real world. If one of the measurements is unknown in the real world, it can be calculated using the cross ratio Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 28 Back to the human height measurement problem If we know the height of one object in the scene We can find the height of a person using vanishing points Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 29 Height measurement with cross-ratio All distances are signed distances Computer Vision - Lecture 09 – Projective Geometry The University of Western Australia 30 Algorithm for calculating height Calculate vertical vanishing point Calculate vanishing line of the reference plane Select top and base points of the reference object Compute metric factor Repeat • Select top and base points of the object to measure • Compute the height of the object A.