• A vector space V consists of a set of vectors – Two vectors can be added – A vector can be multiplied by a scalar – Both operations result again in a vector in V Geometry in Computer Vision – Sets of vectors can be linearly combined into a new vector • The dimension of V = maximal number of vectors which are linear independent Spring 2010 • Basis exists Lecture 1 • Orthogonality between two vectors defined if we have a scalar product Projective Geometry • Linear mappings are well-defined
1 2
A projective space A projective space
• A projective space can be defined from V • The projective space generated from V consist in terms of equivalence classes: of all such projective elements – Any projective element correspond to a 1D subspace – Two vectors u and v are equivalent if there of V – Any projective element has a non-unique exists a non-zero scalar s such that u = s v representation of non-zero vectors in V ⇒ u and v must be non-zero vectors – Any non-zero element of V corresponds to a unique – All vectors which are equivalent correspond to projective element an element of the projective space (a projective element) • The projective space is (here) denoted P(V)
– Projective equivalence is denoted u ∼ v 3 4 A projective space Projective representation
• Dimension of P(V) = dim(V) – 1 •The n-dimensional vector space Rn can be • Addition and scalar multiplications are undefined given a projective representation by the operations in P(V), no linear combinations projective space P(Rn+1) • No basis exists • Orthogonality is well-defined! The homogeneous – Two projective elements are orthogonal iff their coordinates of y representative vectors are orthogonal •A linear mapping M : V U Is represented by the produces a well-defined mapping P(V) P(U) projective element corresponding to
5 6
Example Homogeneous normalization
• Given an a vector u∈Rn+1 we can scale it so that the last element = 1 ⇒ normalization (can we always do this?) • The first n elements in the normalized homogeneous vector are the vector in Rn that u represents • This makes it possible to know which All these vectors in R3 represent the vector in Rn a specific projective element same projective element in P(Rn+1) represents
7 8 Projective representation of the Projective representation of the Euclidean space Euclidean space • The elements of vectors in R2 and R3 are Motivation the coordinates of points in 2D or 3D • A corresponding representation can be found Euclidean spaces relative to some specific also for lines in 2D and lines + planes in 3D. • Operations on these geometric object are much coordinate systems easier to describe algebraically in a projective • We use the projective representation of R2 space than in standard Euclidean coordinates given by P2 = P(R3) – Find the point of intersection between a 3D plane and a 3D line 3 • We use the projective representation of R • “Exceptional cases” can be included in the same given by P3 = P(R4) representations – Example: All 2D lines intersect at one point, except if
9 the lines are parallel or identical 10
A homogeneous representation of A homogeneous representation of lines in 2D lines in 2D
L and are well-defined • Let y = (y1,y2) be the Euclidean also for a vertical line.
coordinates of a 2D point Compare to y2=k y1 + a • Any 2D line is characterized by an angle and a scalar L such that a normal vector (L cos , L sin ) which points from the origin to the line y lies on the line ⇔ y1 cos + y2 sin = L
Origin
The defining equation of a line L = shortest distance from the line to the origin 11 12 A homogeneous representation of A homogeneous representation of lines in 2D lines in 2D • Suggests a homogeneous representation of the line: • y lies on the line ⇔ y1 cos + y2 sin = L
H • y lies on the line ⇔ yH · l = 0 • lH is the (dual) homogeneous coordinates Homogeneous coordinates of y of the line
13 14
Dual homogeneous normalization The cross product
• Given a non-zero vector in R3 we can • The cross product a × b for a, b ∈ R3 determine which line it represents by scaling it such that a × b is orthogonal to a and b – The norm of elements 1 and 2 equals 1 – Third element is non-positive ( 0) a × b = 0 if a = b • The elements of the normalized vector directly gives and L a × b = - (b × a) ∼ (b × a)
15 16 The cross product operator The intersection of two lines
H H • For a fix vector a, the cross product with b • Let l1 and l2 be the dual homogeneous is a linear mapping on b representation of two lines in 2D •The “a ×” mapping can be represented by • Wanted: the intersection point x between the lines an anti-symmetric 3 × 3 matrix [a]×: • Its homogeneous representation yH must satisfy H H yH · l1 = yH · l2 = 0 H H ⇒ yH is orthogonal to both l1 and l2
l 1 y H H such that a b = [a]b yH = l1 × l2
17 18 l2
Special case 1 Special case 2
• If the two lines are identical • If the two lines are distinct but parallel, y is – y is not unique “undefined”, but …
– any point on the line is an intersection point Cannot be normalized H H to represent a 2D point • In this case: l1 × l2 = 0 • We can use the result = 0 to flag that the lines are identical, i.e., the result is not a specific point
Result = 0 multiple solutions exist
19 Assuming the lines are distinct 20 Points at infinity Line intersecting two points
• The result in this case can be used to represent • Let y1H and y2H be the homogeneous a “point at infinity” coordinates of two points in 2D – The normalization suggests that the corresponding • We want to find the line which intersects both 2D point lies at infinite distance from the origin points • This is a single point even though there are two • Its dual homogeneous representation lH must directions to look for this point satisfy – An abstraction of an orientation of a line in 2D H H yH1 · l = yH2 · l = 0 2 • Given “for free” as elements in P H ⇒ l is orthogonal to yH1 and yH2 • The result of an operation which maps onto P2 is y either a proper 2D point or a point at infinity l 2 H l = yH1 × yH2 21 22 y1
Special case 1 Special case 2
• If the two points are identical, the line is • This operation still works also when only not unique: any line going through one one of the two points is a point at infinity: point goes through the other – The resulting line goes through the first point
• In this case: yH1 × yH2 = 0 – In the orientation given by the second point • We can (again) use the result = 0 to flag (the point at infinity) that the points are identical, i.e., the result is not a specific line but rather a set of lines
23 24 Special case 3 The line at infinity
• The operation even works when both • The result in this case can be used to represent points are points at infinity: a “line at infinity” – The normalization suggests that the corresponding 2D line lies at infinite distance from the origin
Cannot be normalized • There is only one single line at infinity to represent a 2D line – Represents the line which intersects with any distinct pair of points at infinity – An abstraction of a circle at infinite distance from the origin – Given “for free” as an element of P2 – The result of an operation which maps onto P2 can be either a proper 2D line or the line at infinity Assuming the points at inifity are distinct 25 26
Notation Affine coordinate transformations
• In the following, most vectors are • A 2D point y is transformed to y’ such that homogeneous representations of points or the corresponding Euclidean 2D lines coordinates are related as – Drop the H –Use y to denote homogeneous coordinates of a 2D point. y is then an element of P2 – The corresponding 2D point is also called y ! –Use l to denote dual homogeneous coordinates of a 2D line. l is then an element of P2 • This transformation is called affine
– The corresponding line is also called l ! 27 28 Affine coordinate transformations Affine coordinate transformations
• Translation: • Rotation:
y2 y’2 y2 y’2
y1 y’1
Note: rotation y1 y’1 29 around the origin! 30
Coordinate transformations Coordinate transformations
• Scaling • Mirroring
y2 y’2 y2 y’2
y1 y’1 y1 y’1
31 32 Affine coordinate transformations Affine coordinate transformations
• Skewing • In homogeneous coordinates:
y2 y’2
y1 y’1 All these transformations are represented as linear 33 mappings T onto the homogeneous coordinates 34
Coordinate transformations 2D coordinate transformations
• The composition of two such matrices is again a • Let y ∈ P2 homogeneous coordinates of a matrix of this type: a matrix group – Rotation around an arbitrary point can be represented 2D point as a composition of translation-rotation-translation • Let T be a 3 × 3 matrix which represents • The 3 × 3 transformation matrix is itself an element of a projective space some coordinate transformation: y’ = T y – A multiplication by a non-zero scalar onto the matrix – Note that y’ represent the same point as y but can be moved to either of the two homogeneous vectors y or y’ which gives equivalent homogeneous in a different coordinate system!! vectors • Let l be a line that includes y: l·y = 0 • The transformation matrix can be more general (why?) than described here • If then follows that l transforms to l’=(TT)-1 l – More on this after the 3D case has been described •(TT)-1 is called the dual transformation of T 35 36 A homogeneous representation of A homogeneous representation of planes in 3D planes in 3D
• Let (x1,x2,x3) be the Euclidean coordinates of a 3D point x • Any 3D plane is characterized by a unit (L n1, Ln2, Ln3) vector n=(n1,n2,n3) and a scalar L such a normal vector that which points from the origin to the plane
x lies on the plane ⇔ x1 n1+x2 n2+x3 n3 = L Origin
L = shortest distance from the plane to the origin The defining equation of a plane 37 38
A homogeneous representation of A homogeneous representation of planes in 3D planes in 3D • Suggests a homogeneous representation of the plane: • x lies on the plane ⇔ x1 n1+x2 n2+x3 n3 = L
• x lies on the plane ⇔ x · p = 0 • p are the (dual) homogeneous coordinates of the plane Homogeneous coordinates of x 39 40 Dual homogeneous normalization Points and planes at infinity
• Given a vector in R4 we can determine Similar to the 2D case: which plane it represents by scaling it such • In 3D there are points at infinity that – Have last homogeneous coordinate = 0 – The norm of elements 1 to 3 equals 1 – Fourth element is non-positive ( 0) • There is a single 3D plane at infinity • The elements of the normalized vector – Intersects all 3D points at infinity directly gives n and L • Similar to the 2D case: x’ = T x ⇔ p’ = (TT)-1 p
41 42
Affine transformations in 3D BREAK!
• Simple extension from the 2D case!!
43 44 A homogeneous representation of Parametric representation of lines lines in 3D in 3D
• Let x1 and x2 be two distinct 3D points with 3 • 3D lines can be represented in several x1, x2 ∈ P slightly different ways • Any point x on the line can be written
• Here we will use x = t x1 + (1 - t) x2 for some t ∈ R – so called Plücker coordinates in the form of an anti-symmetric matrix x – Parametric representation: x2
x = x0 + t n eller x = t x1 + (1 - t) x2 x1 45 46
A homogeneous representation of Intersection between lines in 3D a line and a plane in 3D T T Define L = x1 x2 –x2 x1 • Let L be the Plücker coordinates of a 3D • L is a homogenous representation of the line line which intersects x1 and x2 • Let p the dual homogeneous coordinates • L is a 4 × 4 anti-symmetric matrix: LT =-L of a plane • L can be seen as a projective element (why?) • Which is the intersection point x0? • Referred to as Plücker coordinates of the – Characterized by x0 · p = 0 line L
• As a projective element L is independent x x ∼ L p 0 of which two distinct points on the line are 0 p
used (why?) 47 48 Dual Plücker coordinates 2D coordinates on a 3D plane
• Alternatively, let p1 and p2 be two planes that • The Euclidean coordinates of a 3D point in intersect the 3D line a plane can be described as T T • L’= p1p2 – p2p1 is the dual Plücker coordinates of the line • Independent of which 2 planes we use (as long as they are distinct and intersect the line) • L’x gives the plane that includes the line and point x • Relation between L and L’? Some point on Tangent vectors of the plane the plane
49 All three vectors define a 2D coordinate system in the plane 50
2D coordinates on a 3D plane 2D coordinates on a 3D plane
Homogeneous 2D • In homogeneous coordinates: coordinates: y a
o b
Mapping from 2D to 3D homogeneous or x = P y coordinates: P
Origin of the 3D space P uniquely defines the 2D 51 coordinate system on the plane 52 2D to 2D projective mappings 2D to 2D projective mappings
•Given – two 3D planes, each with its own 2D coordinate x2 b system, P1 and P2 2
– a 3D point n a1 n there is a unique mapping from one plane to the o1
other: a2 o2 x1
Project a point x1 on the first plane through n b1 onto the second plane which gives x2 In this example: n is located in between the planes 53 This is not required! 54
2D to 2D projective mappings Homography
• The geometric relation between x1, x2, and • This mapping on the 2D coordinates in the n together with x1 = P1 y1 and x2 = P2 y2 two planes is more general than the affine leads to (why?) transformations described earlier! • Called homography or projective transformation y2 = Hy1 • Any 3 3 non-singular H is a homography • Describes e.g. how a pinhole-camera • H is a 3 × 3 general non-singular matrix maps points on a plane to the image plane • Depends on the two planes and on n 55 56 Homography Homographies
• Includes the affine transformations! • Any homography is determined by how it – In the special case that the planes are parallel maps 4 distinct points – In other cases: there are points at infinity that are mapped to normal points and vice versa
• We assume that n is not included in any of y2 y2 the two planes ⇒ H is always invertible – We can uniquely go from image coordinates to coordinates in the plane • H always maps a line to a line (why?)
y1 y1 57 58
Homographies 3D homography transformations
• One or two of the 4 points may be at • The group of 4 4 non-singular matrices infinity define the group of 3D homography transformations • Analogue to the 2D case, but cannot be y y 2 2 characterized in terms of projective mappings in a simple way
y1 y1 59 60 The pinhole camera model The pinhole camera model
The image plane and the camera center define a camera-centered Principal or optical axis • If we look at the camera coordinate system coordinate system (x1,x2,x3): along the x2 axis: x1,x2 are parallel to the image plane, x3 is perpendicular to the P=(x1,x2,x3) is plane and defines the viewing a point in 3D direction of the camera space
Q=(y1,y2) is the projection f = focal length, the of P Two similar distance between the image plane and the triangles give: camera center 61 62
The pinhole camera model The virtual image plane
o • Looking along the x1 axis gives a similar • The projected image is rotated 180 expression for y2 relative to how we “see” the 3D world
• This can be summarized as: – Reflection in both y1 and y2 coordinates = rotation • Must be de-rotated before we can view it • Mathematically this is equivalent to placing the image plane in front of the focal point • Called a virtual image plane
63 64 The pinhole-camera The pinhole-camera
• We now have: • Using also homogeneous 3D coordinates: The mapping of 3D coordinates to 2D image coordinates defined by the pinhole-camera in camera centered coordinates (virtual image plane!) • In homogeneous image coordinates
Defines a 3 × 4 matrix C C is the camera (projection) matrix 65 66
The normalized camera The camera center
• In the case of a normalized camera: f = 1 • In the camera centered coordinate system, the camera center (focal point) has 3D coordinates (0,0,0) • The camera matrix maps this point to:
Notation for the normalized camera matrix • The homogeneous representation of the camera center lies in the null space of the
67 camera matrix 68 The general camera matrix The general camera matrix
• The camera matrix defined so far assumes • Assuming that the world coordinate that both 2D and 3D coordinates are given system we use is Euclidean, there is in a camera centered coordinate system always a rotation and translation of the 3D • We want to be able to use coordinate system that align it with the – 3D coordinates in any coordinate system of camera centered system our choice, world coordinates – 2D image coordinates in a pixel based coordinate system, often with the origin at the
top left corner and first coordinate down World origin in Camera centered Rotation 3D world camera centered 69 3D coordinates transformation coordinates coordinates 70
The general camera matrix The general camera matrix
• In homogeneous coordinates: • The normalized image coordinates are then given as
y0 = C0 x’ = C0 Te x
Camera centered Defines the transformation World coordinates 3D coordinates matrix Te
C T = ( R | d ) 71 0 e 72 Image coordinates Image coordinates
• Normalized image coordinates • To transform from y to standard image y 0 –f = 1 2 coordinates y
– Origin at the image center (c1,c2) = coordinates of the image center in the y1 – First coordinate right, second up standard coodinates – Same length unit as in 3D space
• f = focal length • Standard image coordinates • r = pixel resolution [pixel / length unit] y2 – Arbitrary f > 0 Defines the transformation matrix T – Origin at the image top left i – First coordinate down, second right or vice versa 73 74 – Pixel based length unit y1
The general camera matrix The general camera matrix
• We can now summarize all this as • Te depends on where the camera (camera center!) is positioned in 3D space and how it is oriented. May be variable or fixed depending on application y = Ti y0 = Ti C0 Te x = Cx • Ti depends on the type of camera, and its setting such as zoom, resolution, etc. Typically • The general camera matrix C is given by fixed. •Since C is the product of three matrices of Internal (intrinsic) External (extrinsic) camera parameters camera parameters rank 3, 3, and 4 C has rank 3 C = Ti C0 Te • To determine C is referred to as The normalized camera calibration (separate lecture) camera matrix 75 76 Equivalent cameras Affine camera
• Let C1 and C2 be the camera matrices of two • In certain applications the 3D points have pinhole cameras with the same camera center n a distance d to the camera that does not y1 = C1 xC1 n = 0 vary much relative to the distance
y2 = C2 xC2 n = 0 • In this case: there is a homography mapping H
from y1 to y2 defined by C1 and C2 such that -1 • In homogeneous coordinates: y1 = Hy2 y2 = H y1 (why?) The affine camera • The images in the two cameras are identical matrix: it always except for a geometric transformation has bottom row (0 0 0 d) – In practice the images crop different parts! 77 78
The orthographic camera Conics (in 2D)
• An identical case appears when the 3D •(y1, y2) lies on a conic curve centered on points are at a large distance from the the origin if A is 2 2 symmetric camera and determines the • Referred to as an orthographic camera character of the curve y1 y1 y2 A =1 y2 • Note: the affine/orthographic property is derived from propoerties of the 3D points, not of the camera
79 80 Conics (in 2D) Conics (in 2D)
• In homogeneous coordinates the defining Assuming that y lies on the conic equation becomes • We can interpret Q y as a line that must pass through y (why?) A0 yT y = yT Qy=0 • This line is in fact the tangent lt of the 0T 1 µ − ¶ conic at point y Q is 3 3 symmetric l • Generalizes to conics at arbitrary positions t
by appropriate translations y
81 82
Dual conics
• yTQy= 0 defines the points y that lie on a conic • Follows: lT Q-1 l = 0 defines the lines that are tangent to the same conic (why?) • Q-1 is the dual conic relative to Q • Q-1l gives the tangent point of tanget line l
83