Announcements

• Assignment 0: “Getting Started with Image Formation Matlab” is posted to web page, due and Cameras Thursday • Read Chapters 1 & 2 of Forsyth & Ponce Computer Vision I • Add yourself to mailinglist CSE 252A Lecture 3

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

Image Formation: Outline Earliest Surviving Photograph • Factors in producing images • Projection • Perspective/Orthographic Projection • Vanishing points • Projective Geometry • Rigid Transformation and SO(3) •Lenses •Sensors • Quantization/Resolution • First photograph on record, “la table service” by • Illumination Nicephore Niepce in 1822. • Reflectance and Radiometry • Note: First photograph by Niepce was in 1816. CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

How Cameras Produce Images Images are two-dimensional patterns of brightness values.

• Basic process: – photons hit a detector – the detector becomes charged – the charge is read out as brightness

• Sensor types: – CCD (charge-coupled device) • high sensitivity • high power • cannot be individually addressed • blooming –CMOS • simple to fabricate (cheap) • lower sensitivity, lower power

• can be individually addressed Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969.

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 They are formed by the projection of 3D objects. Computer Vision I

1 Effect of Lighting: Monet Change of Viewpoint: Monet

Haystack at Chailly at sunrise (1865)

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

Pinhole Camera: Perspective projection Camera Obscura • Abstract camera model - box with a small hole in it

"When images of illuminated objects ... penetrate through a small hole into a very dark room ... you will see [on the opposite wall] these objects in their proper form and color, reduced in size ... in a reversed position, owing to the intersection of the rays". Da Vinci http://www.acmi.net.au/AIC/CAMERA_OBSCURA.html (Russell Naughton) CS252A, Winter 2006 Forsyth&Ponce Computer Vision I CS252A, Winter 2006 Computer Vision I

Camera Obscura Camera Obscura

Jetty at Margate England, • Used to observe eclipses (e.g., Bacon, 1214-1294) 1898. • By artists (e.g., Vermeer).

http://brightbytes.com/cosite/collection2.html (Jack and Beverly Wilgus)

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

2 Purely Geometric View of Perspective Distant objects are smaller

The projection of the point P on the image plane Π’ is given by the point of intersection P’ of the ray defined by PO the plane Π’.

CS252A, Winter 2006 (Forsyth & Ponce) Computer Vision I CS252A, Winter 2006 Computer Vision I

Geometric properties of projection Equation of Perspective Projection

• 3-D Points map to points • 3-D Lines map to lines • Planes map to whole image or half-plane • Polygons map to polygons • Angles & distances not preserved

• Degenerate cases: Cartesian coordinates: x y – line through focal point project to point • We have, by similar triangles, that ( x , y , z ) → ( f , f ) – plane through focal point projects to a line (x, y, z) -> (f x/z, f y/z, f) z z • Ignore the third coordinate, and get CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

What is the intersection of two lines in a plane?

A Digression

Projective Geometry A Point and Homogenous Coordinates

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

3 Do two lines in the plane always Can the perspective image of two parallel intersect at a point? lines meet at a point? YES

No, Parallel lines don’t meet at a point.

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

Projective Geometry • Axioms of Projective Plane Projective geometry provides an elegant means for handling these different 1. Every two distinct points define a line situations in a unified way and 2. Every two distinct lines define a point (intersect at a point) homogenous coordinates are a way to represent entities (points & lines) in 3. There exists three points, A,B,C such that C does not lie on the line defined by A and B. projective spaces. • Different than Euclidean (affine) geometry • Projective plane is “bigger” than affine plane – includes “line at infinity”

Projective Affine Line at Plane = Plane + Infinity

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

Homogenous coordinates Homogenous coordinates A way to represent points in a projective space A way to represent points in a projective space 1. Use three numbers to represent a 1. Use three numbers to represent a point on a projective plane point on a projective plane 2. Add an extra coordinate • Why do this? 2. Add an extra coordinate e.g., (x,y) -> (x,y,1) – Possible to represent e.g., (x,y) -> (x,y,1) points “at infinity” Impose equivalence relation (x,y,1) 2. Impose equivalence relation • Where parallel lines (x,y,z) ≈ λ*(x,y,z) (x,y,z) ≈ λ*(x,y,z) intersect Z • Where parallel planes such that (λ not 0) 1 such that (λ not 0) Y (x,y) intersect i.e., (x,y,1) ≈ (λx, λy, λ) i.e., (x,y,1) ≈ (λx, λy, λ) – Possible to write the action of a perspective 3. Point at infinity – zero for last X coordinate camera as a matrix e.g., (x,y,0)

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

4 Conversion Euclidean -> Homogenous -> Points at infinity

In 2-D Euclidean Point at infinity – zero for last coordinate (x,y,0) (x,y,1) • Euclidean -> Homogenous: and equivalence relation (x, y) -> k (x,y,1) Z (x,y,0) ≈ λ*(x,y,0) 1 • Homogenous -> Euclidean: Y (x, y, z) -> (x/z, y/z) (x,y,1) No corresponding Euclidean (x,y,0) Z point 1 X In 3-D Y (x,y) • Euclidean -> Homogenous: (x, y, z) -> k (x,y,z,1) X • Homogenous -> Euclidean: Projective Affine Line at (x, y, z, w) -> (x/w, y/w, z/w) Plane = Plane + Infinity CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

Lines in Projective space Projective transformation

1. Line in Euclidean plane • 3 x 3 linear transformation of homogenous 2. Plane in homogenous coordinates coordinates • Points map to points, 3. Plane is represented by its • lines map to lines normal N 4. Equation for plane is Z N . (x,y,z) = 0 1 ⎡u1 ⎤ ⎡a11 a12 a21 ⎤⎡x1 ⎤ Y M . (x,y,z) = 0 ⎢u ⎥ = ⎢a a a ⎥⎢x ⎥ ⎢ 2 ⎥ ⎢ 21 22 23 ⎥⎢ 2 ⎥ Where M=lamda * N N ⎣⎢u3 ⎦⎥ ⎣⎢a31 a32 a33 ⎦⎥⎣⎢x3 ⎦⎥ X

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

The equation of projection

End of the Digression

Homogenous Coordinates and Camera matrix Cartesian coordinates: ⎛ ⎞⎛ X⎞ ⎛ U ⎞ 10 0 0⎜ ⎟ ⎜ ⎟ Y x y ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ V ⎟ = 01 0 0⎜ ⎟ (x,y,z) → ( f , f ) ⎜ ⎟ ⎜ ⎟ Z z z ⎝W ⎠ ⎜ 00 1 0⎟⎜ ⎟ ⎝ f ⎠⎝ T ⎠ CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

5 Parallel lines meet in the image Vanishing points

• vanishing point

H VPL VPR

VP VP1 2

To different directions Image plane correspond different vanishing points CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 VP3 Computer Vision I

Vanishing Points Vanishing Point

• In the projective plane, parallel lines meet at a point at infinity.

• The vanishing point is the perspective projection of that point at infinity, resulting from multiplication by the camera matrix.

CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

Affine Camera Model Perspective ⎡u⎤ f ⎡x⎤ ⎢ ⎥ = ⎢ ⎥ • Take Perspective projection equation, ⎣v⎦ z ⎣y⎦ and perform Taylor Series Expansion Assume that f=1, and perform a Taylor series expansion about (x0, y0, z0) about (some point (x0,y0,z0). ⎡u⎤ 1 ⎡x0 ⎤ 1 ⎡1⎤ 1 ⎡0⎤ • Drop terms of higher order than linear. = + ()x − x0 + ⎢ ⎥()y − y0 ⎢v⎥ ⎢y ⎥ ⎢ ⎥ z 1 ⎣ ⎦ z0 ⎣ 0 ⎦ z0 ⎣0⎦ 0 ⎣ ⎦ • Resulting expression is the affine 1 ⎡x0 ⎤ 1 2 ⎡x ⎤ − ()z − z + 0 ()z − z 2 + z 2 ⎢y ⎥ 0 3 ⎢ ⎥ 0 L camera model 0 ⎣ 0 ⎦ 2 z0 ⎣y0 ⎦ Dropping higher order terms and regrouping. ⎡x⎤ ⎡u⎤ 1 ⎡x ⎤ ⎡1/ z 0 − x / z 2 ⎤ 0 0 0 0 ⎢y⎥ Ap b ⎢ ⎥ ≈ ⎢ ⎥ + ⎢ 2 ⎥ = + v z y 0 1/ z − y / z ⎢ ⎥ ⎣ ⎦ 0 ⎣ 0 ⎦ ⎣ 0 0 0 ⎦⎢ ⎥ CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 ⎣z⎦ Computer Vision I

6 Properties of Affine or Orthographic Orthographic projection camera models

Take Taylor series about (0, 0, z0) – a point on optical axis • Parallel lines in the world, project to parallel lines. • Ratios of distances are preserved • Ratios of angles are preserved.

⎛ X ⎞ ⎛U ⎞ ⎛1 0 0 0⎞⎜ ⎟ ⎧x'= x ⎜ ⎟ ⎜ ⎟⎜ Y ⎟ ⎨ ⎜ V ⎟ = ⎜0 1 0 0⎟ y'= y ⎜ Z ⎟ ⎩ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎝W ⎠ ⎝0 0 0 1⎠⎜ ⎟ ⎝ T ⎠ 3 by 4 camera matrix CS252A, Winter 2006 Computer Vision I CS252A, Winter 2006 Computer Vision I

7