Single View Geometry Camera Model & Orientation + Position Estimation
Total Page:16
File Type:pdf, Size:1020Kb
Single View Geometry Camera model & Orientation + Position estimation What am I? Vanishing point Mapping from 3D to 2D Point & Line Goal: Homogeneous coordinates Point – represent coordinates in 2 dimensions with a 3-vector &x# &x# homogeneous coords $y! $ ! ''''''→$ ! %y" %$1"! The projective plane • Why do we need homogeneous coordinates? – represent points at infinity, homographies, perspective projection, multi-view relationships • What is the geometric intuition? – a point in the image is a ray in projective space y (sx,sy,s) (x,y,1) (0,0,0) z x image plane • Each point (x,y) on the plane is represented by a ray (sx,sy,s) – all points on the ray are equivalent: (x, y, 1) ≅ (sx, sy, s) Projective Lines Projective lines • What does a line in the image correspond to in projective space? • A line is a plane of rays through origin – all rays (x,y,z) satisfying: ax + by + cz = 0 ⎡x⎤ in vector notation : 0 a b c ⎢y⎥ = [ ]⎢ ⎥ ⎣⎢z⎦⎥ l p • A line is also represented as a homogeneous 3-vector l Line Representation • a line is • is the distance from the origin to the line • is the norm direction of the line • It can also be written as Example of Line Example of Line (2) 0.42*pi Homogeneous representation Line in Is represented by a point in : But correspondence of line to point is not unique We define set of equivalence class of vectors in R^3 - (0,0,0) As projective space Projective lines from two points Line passing through two points Two points: x Define a line l is the line passing two points Proof: Line passing through two points • More specifically, Matlab codes • function l = get_line_by_two_points(x, y) • x1 = [x(1), y(1), 1]'; • x2 = [x(2), y(2), 1]'; • l = cross(x1, x2); • l = l / sqrt(l(1)*l(1)+l(2)*l(2)); Example of Line Verify p2 lies on the line P2 >> 218*0.6+13*0.8 ans = 141.2000 (close) P1 Example of Line P2 Test line: >> p1 = [42,142,1]; >> p2=[218,13,1]; >> l = cross(p1,p2) l = 129 176 -30410 P1 >> l = l/sqrt(l(1)^2+l(2)^2) l = 0.5912 0.8066 -139.3591 >> x = [221;23]; >> y = [218;268]; >> l = get_line_by_two_points(x,y); >> l l = -0.2448 -0.9696 265.4744 Projective lines from two points When does the line has the form (a, b, 0)? -all lines passing through the optical center When does the line has the form (0, 0, 1)? -line at infinity Points from two lines Intersection of lines Given two lines: l , Define a point X is the intersection of the two lines Intersection of lines • More specifically, Matlab codes • function x0 = get_point_by_two_line(l, l1) • x0 = cross(l, l1); • x0 = [x0(1)/x0(3); x0(2)/x0(3)]; Example of Lines Intersection Point and line duality – A line l is a homogeneous 3-vector – It is ⊥ to every point (ray) p on the line: l p=0 p p2 l1 p l 1 l2 What is the line l spanned by rays p1 and p2 ? • l is ⊥ to p1 and p2 ⇒ l = p1 × p2 • l is the plane normal What is the intersection of two lines l1 and l2 ? • p is ⊥ to l1 and l2 ⇒ p = l1 × l2 Points and lines are dual in projective space • given any formula, can switch the meanings of points and lines to get another formula Points from two lines When P has the form (x,y,0)? Point at infinity Example: Consider two parallel horizontal lines: Intersection = Point at infinity in the direction of y Point at infinity, Ideal points Intersection: Any point (x1,x2,0) is intersection of lines at infinity Points at infinity • Under projective transformation, – All parallel lines intersects at the point at infinity – One point at infinity ó one parallel line direction • Where are the points at infinity in the image plane? Line at infinity • A line passing all points at infinity: • Because : Ideal points and lines (a,b,0) y (sx,sy,0) y z image plane z image plane x x • Ideal point (“point at infinity”) – p ≅ (x, y, 0) – parallel to image plane – It has infinite image coordinates Ideal line • l ≅ (a, b, 0) – parallel to image plane • Corresponds to a line in the image (finite coordinates) Comparing heights Vanishing Point Measuring height 5.4 5 Camera height 4 3.3 3 2.8 2 1 Measuring height vz r vanishing line (horizon) t0 t vx v vy H R H b0 b Focal Length • A Lighting Reproduction Approach to Live-Action Compositing, P. Debevec 2002 FOV depends of Focal Length f f Smaller fOV = larger focal Length Field of View (Zoom) Field of View (Zoom) Large Focal Length compresses depth 400 mm 200 mm 100 mm 50 mm 28 mm 17 mm © 1995-2005 Michael Reichmann http://www.creativepro.com/img/story/102604_fg3b.jpg Nikon’s 1200-1700mm f/5.6-8P lens. Nikon’s 1200-1700mm f/5.6-8P lens. " /012'+$3+/4$&'()5$2 35'0:+$3 +>5'?+!6 ;<=6 $%&'()*+,+-+. !! ;<=66 36+!+3$(10+0'27)8 # 366+!+3$(10+0'27)8 9.+!+:5*)12('+)$+$%&'()+. 9,+!+:5*)12('+)$+$%&'()+, Canon D30 takes image with 2160x1440 pixels. What is the pixel size? What f to make tower appear half the size of image? Fix f=50mm, D2 = ? in order to make the tower 1/3 size of image? D1=? to make human the same height as tower? What f’ to make tower appear twice as big as before (2/3 the height)? Setting f’=100mm, D1=? in order to keep the original size of human? (1/3 the height) Lens distortion with focal length Lens distortion with focal length Camera projection model The overall goal is to compute 3D geometry of the scene from just 2D images. We will first study 3D->2D camera projection using linear algebra. The Pinhole Camera – Light enters a darkened chamber through a pinhole opening and forms an image on the further surface The Pinhole Camera Model Ideal case: yc y Projection equation: y’ x’ = f X / Z p C y’ = f Y / Z z zc f xc Zx’ = f X Zy’ = f Y Z = Z Step 1: Camera projection matrix Zx’ = f X Zy’ = f Y y Z = Z c y y’ p C f 0 0 0 Xc x’ z Z zc f 0 f 0 0 Yc = y’ c Z 0 0 1 0 c 1 x 1 c P0 X = x Step 2: Intrinsic camera parameters: map camera coordinate to pixel coordinate yc αx s px x’ x 0 p y’ y αy y = y 0 0 1 z’ z y’ p C z zc f K Optical world Pixel world (3x3 submatrix) xc αx, αy is pixel scaling factor px, py is the principle point (where optical axis hits image plane) s is the slant factor, when the image plane is not normal to the optical axis Combine the Intrinsic camera parameters X αx s px f 0 0 0 c x Y 0 αy py 0 f 0 0 c = y 0 0 1 0 0 1 0 Zc 1 1 x αxf s px 0 c x y 0 αyf py 0 c = y 0 0 1 0 zc z 1 K (Calibration matrix) P0 X = x P = [K , 0] = K [I, 0] Step 3: External parameters: rotation and translation map the world to camera coordinates 3x3 3x1 (R,t) Camera coordinate yc World coordinate y y’ p C z zc f xc Combining Internal and External parameters (R,t) Camera coordinate 1) yc World coordinate y y’ p C z zc f 2) xc 1) Translate the world coordinate into the camera coordinate 2) Translate the camera coordinate into the pixel coordinate Combining Internal and External parameters (R,t) Camera coordinate yc World coordinate y y’ p C z After simplication: zc f xc x = K [R, t] X pixel world Special case, planar world, homograph (R,t) X= (xw,yw,zw=0,1) yc World coordinate y y’ p C x = K [R, t] X z zc f Expand: xc xw Camera coordinate yw x = K r1 r2 r3 t zw=0 1 Special case, planar world, homograph (R,t) X= (xw,yw,zw=0,1) yc World coordinate y After simplication: y’ p C z zc f w x xc x = K r1 r2 t yw Camera coordinate 1 We have the homographic mapping: X = K H3x3 X Special case: rotating camera (R,t) x = K [R, t] X yc World coordinate with t = 0 y y’ p C Expand: z x = K [R] X zc f xc Camera coordinate xw w This is also a homography with X = y H = R zw Panoramas x = K [R] X Visual Odometry: Overview 1) The transformation (R,t) tells us (R,t) how to rotate + move the world coordinate to the camera yc World coordinate. coordinate y 2) In the following slides, we will see y’ p C z 1) how to get the world reference’s zc f frame orientation+position from (R,t), xc 2) how to get the pan/tilt/yaw angles from R 3) If we want to know the camera orientation+position, we need to look at its inverse transform, which we should see is , 1) The computation of the camera pan/tilt/yaw angle is the same as in the previous case Recover Camera orientation Case 1: single vanishing point (z- direction) Pan/tilt/yaw angles Pan : rotation Yaw :rotation Tilt : rotation around y-axis around z-axis around x-axis z y y x x z Rotation: Pan/Tilt/Yaw Given the 3x3 rotation matrix R, we can recover, pan/tilt/yaw angle, and vise.