Announcements

• Assignment 0: “Getting Started with Image Formation, Matlab” is due today Cameras (cont.) • Read Chapters 1 & 2 of Forsyth & Ponce

Computer Vision I CSE 252A Lecture 4

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

The course Pinhole Camera: Perspective projection • Part 1: The physics of imaging • Abstract camera model - box • Part 2: Early vision with a small hole in it • Part 3: Reconstruction • Part 4: Recognition

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Forsyth&Ponce Computer Vision I

Equation of Perspective Projection Geometric properties of projection • 3-D points map to points • 3-D lines map to lines • Planes map to whole image or half-plane • Polygons map to polygons

• Important point to note: Angles & distances not preserved, nor are inequalities of angles & distances. Cartesian coordinates: • We have, by similar triangles, that (x, y, z) -> (f’ x/z, f’ y/z, f’) • Degenerate cases: • Establishing an image plane coordinate system at C’ aligned with i – line through focal point project to point and j, we get – plane through focal point projects to a line CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

1 Homogenous coordinates

• Our usual coordinate system is called a Euclidean or affine coordinate system

A Digression • Rotations, translations and projection in Homogenous coordinates can be expressed linearly as matrix multiplies

Projective Geometry Projection Convert Convert and Homogenous Coordinates Euclidean Homogenous Homogenous Euclidean World World Image World 3D 3D 2D 2D

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

What is the intersection of Do two lines in the plane always two lines in a plane? intersect at a point?

No, Parallel lines don’t A Point meet at a point.

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

Can the perspective image of two parallel lines meet at a point? Projective geometry provides an elegant YES means for handling these different situations in a unified way and homogenous coordinates are a way to represent entities (points & lines) in projective spaces.

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

2 Homogenous coordinates Projective Geometry A way to represent points in a projective space • Axioms of Projective Plane • Use three numbers to represent a point on a 1. Every two distinct points define a line projective plane 2. Every two distinct lines define a point (intersect Why? The projective plane has to be bigger • Why do this? than the Cartesian plane. at a point) – Possible to represent 3. There exists three points, A,B,C such that C How: Add an extra coordinate points “at infinity” does not lie on the line defined by A and B. e.g., (x,y) -> (x,y,1) • Where parallel lines intersect Impose equivalence relation • Different than Euclidean (affine) geometry • Where parallel planes (x,y,z) ≈ λ*(x,y,z) intersect • Projective plane is “bigger” than affine such that (λ not 0) – Possible to write the plane – includes “line at infinity” i.e., (x,y,1) ≈ (λx, λy, λ) action of a perspective camera as a matrix Projective Affine Line at • Point at infinity – zero for last coordinate Plane = Plane + Infinity e.g., (x,y,0)

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

Homogenous coordinates Conversion A way to represent points in a projective space Euclidean -> Homogenous -> Euclidean Use three numbers to represent a In 2-D point on a projective plane • Euclidean -> Homogenous: (x, y) -> k (x,y,1) Add an extra coordinate • Homogenous -> Euclidean: e.g., (x,y) -> (x,y,1) (x,y,1) (x, y, z) -> (x/z, y/z) (x,y,1) Z Z Impose equivalence relation 1 1 Y (x,y) In 3-D Y (x,y) (x,y,z) ≈ λ*(x,y,z) • Euclidean -> Homogenous: such that (λ not 0) (x, y, z) -> k (x,y,z,1) i.e., (x,y,1) ≈ (λx, λy, λ) X X • Homogenous -> Euclidean: (x, y, z, w) -> (x/w, y/w, z/w)

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

Points at infinity Lines in Projective space

Point at infinity – zero for 1. Line in Euclidean plane last coordinate (x,y,0) (x,y,1) 2. Plane through origin in and equivalence relation homogenous coordinates Z (x,y,0) *(x,y,0) ≈ λ 1 3. Plane is represented by its Y normal N No corresponding Euclidean (x,y,0) 4. Equation for plane is Z point N . (x,y,z) = 0 1 X Y M . (x,y,z) = 0 Where M=lamda * N N X Projective Affine Line at Plane = Plane + Infinity CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

3 Projective transformation The equation of projection

• 3 x 3 linear transformation of homogenous coordinates • Points map to points, • lines map to lines

Homogenous Coordinates and Camera matrix Cartesian coordinates: ⎛ X⎞ ⎛ ⎞ ⎛ ⎞ U ⎜ 1 0 0 0⎟⎜ ⎟ ⎜ ⎟ ⎜ Y ⎟ ⎜ V ⎟ = ⎜ 0 1 0 0⎟ ⎜ ⎟ ⎜ ⎟⎜ Z ⎟ ⎝W ⎠ ⎜ 0 0 1 0⎟⎜ ⎟ ⎝ f ⎠⎝ T ⎠

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 € Computer Vision I

Parallel lines meet in the image Vanishing point

End of the Digression

Image plane

• Formed by line through O • parallel to the given line(s) • A single line can have a vanishing point

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

Appropriate Simplified Camera Models Affine Camera Model in Neighborhood About (x0,y0,z0) Perspective Projection

Affine Camera Model

Scaled • Take perspective projection equation, and perform Orthographic Taylor series expansion about some point (x ,y ,z ). Projection 0 0 0 • Drop terms that are higher order than linear. Orthographic • Resulting expression is affine camera model Projection CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

4 • Perspective Affine Camera Model

• Assume that f=1, and perform a Taylor series

expansion about (x0, y0, z0) Rewrite affine camera model in terms of Homogenous Coordinates

• Dropping higher order terms and regrouping.

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

The projection matrix for scaled Scaled Orthographic projection orthographic projection Starting with Affine camera mode

Take Taylor series about a point on optical axis (0,0,z0), i.e. x0=y0=0 ⎛ X⎞ ⎛ U ⎞ ⎛1 /z0 0 0 0⎞⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ Y ⎟ ⎜ V ⎟ = ⎜ 0 1/z0 0 0⎟ ⎜ ⎟ ⎜ ⎟⎜ Z⎟ ⎝W ⎠ ⎝ 0 0 0 1⎠⎜ ⎟ ⎝ T ⎠

(0, 0, z0)

Parallel lines project to parallel lines Ratios€ of distances are preserved under orthographic

This yields:

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

What if camera coordinate system differs Other camera models from object coordinate system • Generalized camera – maps points lying on rays {c} and maps them to points on the image plane.

Omnicam (hemispherical) Light Probe (spherical) P {W}

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

5 Euclidean Coordinate Systems Coordinate Changes: Pure Translations

B A B OBP = OBOA + OAP , P = P + OA

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

Coordinate Changes: Coordinate Changes: Rigid Transformations Pure Rotations

⎡ A x⎤ ⎡ B x⎤ ⎢ A ⎥ ⎢ B ⎥ OP = [iA jA kA ]⎢ y⎥ = [iB jB kB ]⎢ y⎥ A B ⎣⎢ z⎦⎥ ⎣⎢ z⎦⎥

B B A ⇒ P=A R P CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

€

A convenient notation Some points about SO(n)

nxn T • SO(n) = { R ∈ ℜ : R R = I, det(R) = 1} – SO(2): rotation matrices in plane ℜ2 – SO(3): rotation matrices in 3-space ℜ3 • Forms a Group under matrix product operation: A – Points: P1 – Identity – Leading superscript coordinate system w.r.t. – Inverse – Associative – Subscript – an identifier – Closure – Rotation Matrices • Closed (finite intersection of closed sets) – Lower left (Going from this system) • Bounded Ri,j ∈ [-1, +1] – Upper left (Going to this system) • Does not form a vector space. – To add vectors, coordinate systems must agree • Manifold of dimension n(n-1)/2 – To rotate a vector, points coodrinate system must agree with – Dim(SO(2)) = 1 lower left of rotation matrix – Dim(SO(3)) = 3

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

6 SO(3) Rigid Transformations as Mappings: Rotation about the k Axis • Parameterizations of SO(3) • 3-D manifold, so between 3 parameters and 2n +1 parameters (Whitney’s Embedding Thm.) – Roll-Pitch-Yaw – Euler Angles – Axis Angle (Rodrigues formula) – Cayley’s formula – Matrix Exponential – Quaternions (four parameters + one constraint) = rot(k,θ)

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

Rotation: Homogenous Coordinatesy Rotation p' • About z axis x' 1 0 0 0 x θ p y' 0 cos θ -sin θ 0 y • About = x z rot(z,θ) x axis: z' 0 sin θ cos θ 0 z 1 0 0 0 1 1 x' cos θ -sin θ 0 0 x y' sin θ cos θ 0 0 y x' cos θ 0 sin θ 0 x = y' 0 1 0 0 y z' 0 0 1 0 z • About = 1 0 0 0 1 1 y axis: z' -sin θ 0 cos θ 0 z 1 0 0 0 1 1

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

Rotate(k, θ) Roll-Pitch-Yaw Rotation y θ • About (kx, ky, kz), a unit vector on an arbitrary axis k x (Rodrigues Formula) z

x' kxkx(1-c)+c kzkx(1-c)-kzs kxkz(1-c)+kys 0 x

y' kykx(1-c)+kzs kzkx(1-c)+c kykz(1-c)-kxs 0 y = z' kzkx(1-c)-kys kzkx(1-c)-kxs kzkz(1-c)+c 0 z Euler Angles 1 0 0 0 1 1 where c = cos θ & s = sin θ

CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

7 Quaternions Unit Quaternions and Rotations • Let R denote the rotation of angle θ about the unit vector u. q = (a,α) • Define unit quaternion q = (cos θ/2, sin θ/2 u). • Note |q| = 1 (i.e., q lies on unit sphere for any u and θ. Operations on quaternions: • Sum of quaternions: ( a,α ) + ( b,β ) ≡ (( a+b ),(α+β ) • Then for any vector α, R α = imaginary(q α* q’) • Multiplication by a scalar: λ ( a,α ) ≡ ( λa,λα ) where α*= (0, α) • q and –q define the same rotation matrix. • Quaternion product: If q = (a, ( b, c, d )T) is a unit quaternion, the corresponding ( a,α ) ( b,β ) ≡ (( a b – α ⋅ β ) , ( a β + b α + α × β )) rotation matrix is: • Conjugate: q = (a,α) q’ ≡ (a, -α)

• Norm: | q |2 ≡ q q’ = q’ q = a2 + | α |2 CS252A, Fall 2010 Computer Vision I CS252A, Fall 2010 Computer Vision I

Block Matrix Multiplication Camera parameters • Issue – World units (e.g., cm), camera units (pixels) – camera may not be at the origin, looking down the z-axis What is AB ? • extrinsic parameters – one unit in camera coordinates may not be the same as one unit in world coordinates • intrinsic parameters - focal length, principal point, aspect ratio, angle between axes, etc.

Homogeneous Representation of Rigid Transformations ⎛ X⎞ ⎛ U ⎞ ⎛ Transformation ⎞⎛ 1 0 0 0⎞⎛ Transformation ⎞⎜ ⎟ B A B B A B A B ⎜ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎟ Y ⎡ P⎤ ⎡ P⎤ ⎡ R O ⎤⎡ P⎤ ⎡ R P + O ⎤ V = representing 0 1 0 0 representing ⎜ ⎟ B A A A A ⎜ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟ ⎢ ⎥ = AT⎢ ⎥ = ⎢ ⎥⎢ ⎥ = ⎢ ⎥ ⎜ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎟ Z 1 1 0T 1 1 1 ⎝W ⎠ ⎝ intrinsic parameters⎠⎝ 0 0 1 0⎠⎝ extrinsic parameters⎠⎜ ⎟ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦⎣ ⎦ ⎣ ⎦ ⎝ T ⎠ 3 x 3 4 x 4: Rigid transformation CS252A, Fall 2010 4x4 matrix defines a rigid transformationComputer Vision I CS252A, Fall 2010 Computer Vision I € €

Camera Calibration

, estimate intrinsic and extrinsic camera parameters

• See Text book for how to do it. • Camera Calibration Toolbox for Matlab (Bouguet) http://www.vision.caltech.edu/bouguetj/calib_doc/

CS252A, Fall 2010 Computer Vision I

8