Image Formation
Jana Kosecka
3-D Euclidean Space - Vectors
A “free” vector is defined by a pair of points :
Coordinates of the vector :
1 3D Rotation of Points – Euler angles Rotation around the coordinate axes, counter-clockwise:
P ’ ’ Y γ y P X’ x z
Rotation Matrices in 3D
• 3 by 3 matrices • 9 parameters – only three degrees of freedom • Representations – either three Euler angles • or axis and angle representation
• Properties of rotation matrices (constraints between the elements)
2 Rotation Matrices in 3D
• 3 by 3 matrices • 9 parameters – only three degrees of freedom • Representations – either three Euler angles • or axis and angle representation
• Properties of rotation matrices (constraints between the elements)
Columns are orthonormal
Canonical Coordinates for Rotation
Property of R
Taking derivative
Skew symmetric matrix property
By algebra
By solution to ODE
3 3D Rotation (axis & angle)
Solution to the ODE
with
or
Rotation Matrices
Given
How to compute angle and axis
4 3D Translation of Points
Translate by a vector
P ’ t ’ Y x’ x P z’ y z
Rigid Body Motion – Homogeneous Coordinates
3-D coordinates are related by: Homogeneous coordinates:
Homogeneous coordinates are related by:
5 Rigid Body Motion – Homogeneous Coordinates
3-D coordinates are related by: Homogeneous coordinates:
Homogeneous coordinates are related by:
Properties of Rigid Body Motions
Rigid body motion composition
Rigid body motion inverse
Rigid body motion acting on vectors
Vectors are only affected by rotation – 4th homogeneous coordinate is zero
6 Rigid Body Transformation
Coordinates are related by: Camera pose is specified by:
Image Formation
• If the object is our lens the refracted light causes the images
• How to integrate the information from all the rays being reflected from the single point on the surface ?
• Depending in their angle of incidence, some are more refracted then others – refracted rays all meet at the point – basic principles of lenses
• Also light from different surface points may hit the same lens point but they are refracted differently - Kepler’s retinal theory
7 Let’s design a camera
• Idea 1: put a piece of film in front of an object • Do we get a reasonable image?
Slide by Steve Seitz
Pinhole camera
• Add a barrier to block off most of the rays – This reduces blurring – The opening is known as the aperture
Slide by Steve Seitz
8 Pinhole camera model
• Pinhole model: – Captures pencil of rays – all rays through a single point – The point is called Center of Projection (focal point) – The image is formed on the Image Plane
Slide by Steve Seitz
Camera Obscura
• Basic principle known to Mozi (470-390 BCE), Aristotle (384-322 BCE) • Drawing aid for artists: described by Leonardo da Vinci (1452-1519)
Gemma Frisius, 1558
Source: A. Efros
9 Home-made pinhole camera
Why so blurry?
Slide by A. Efros http://www.debevec.org/Pinhole/
Shrinking the aperture
• Why not make the aperture as small as possible? – Less light gets through – Diffraction effects…
Slide by Steve Seitz
10 Shrinking the aperture
Adding a lens
• A lens focuses light onto the film – Thin lens model: • Rays passing through the center are not deviated (pinhole projection model still holds)
Slide by Steve Seitz
11 Adding a lens
• A lens focuses light onto the film – Thin lens model: • Rays passing through the center are not deviated (pinhole projection model still holds) • All parallel rays converge to one point on a plane located at the focal length f
Slide by Steve Seitz
Adding a lens
• A lens focuses light onto the film – There is a specific distance at which objects are “in focus” • other points project to a “circle of confusion” in the image
Slide by Steve Seitz
12 Thin lens formula
Similar triangles everywhere! y’/y = D’/D
D’ D f y y’
image lens object plane Slide by Frédo Durand
Thin lens formula
Similar triangles everywhere! y’/y = D’/D y’/y = (D’-f)/f D’ D f y y’
image lens object plane Slide by Frédo Durand
13 Thin lens formula Any point satisfying the thin lens 1 + 1 = 1 D’ D f equation is in focus.
D’ D f
image lens object plane Slide by Frédo Durand
Depth of Field
• in real camera lenses, there is a range of points which are brought into focus at the same distance • depth of field of the lens , as Z gets large – z’ approaches f • human eye – power of accommodation – changing f
14 How can we control the depth of field?
• Changing the aperture size affects depth of field – A smaller aperture increases the range in which the object is approximately in focus – But small aperture reduces amount of light –
need to increase exposure Slide by A. Efros
Varying the aperture
Large aperture = small DOF Small aperture = large DOF Slide by A. Efros
15 Field of View
Slide by A. Efros
Field of View
What does FOV depend on? Slide by A. Efros
16 Field of View
f f
FOV depends on focal length and size of the camera retina
Smaller FOV = larger Focal Length Slide by A. Efros
Field of View / Focal Length
Large FOV, small f Camera close to car
Small FOV, large f Camera far from the car Sources: A. Efros, F. Durand
17 Same effect for faces
wide-angle standard telephoto
Source: F. Durand
Image Formation – Perspective Projection
“The School of Athens,” Raphael, 1518
18 Pinhole Camera Model
Pinhole
Frontal pinhole
More on homogeneous coordinates
In homogenous coordinates – these represent the Same point in 3D
The first coordinates can be obtained from the second by division by W
What if W is zero ? Special point – point at infinity – more later
In homogeneous coordinates – there is a difference between point and vector
19 Pinhole Camera Model • Image coordinates are nonlinear function of world coordinates • Relationship between coordinates in the camera frame and sensor plane 2-D coordinates
Homogeneous coordinates
Image Coordinates • Relationship between coordinates in the sensor plane and image
metric coordinates
Linear transformation
pixel coordinates
20 Calibration Matrix and Camera Model • Relationship between coordinates in the camera frame and image Pinhole camera Pixel coordinates
Calibration matrix (intrinsic parameters)
Projection matrix
Camera model
Calibration Matrix and Camera Model • Relationship between coordinates in the world frame and image Pinhole camera Pixel coordinates
More compactly
Transformation between camera coordinate Systems and world coordinate system
21 Radial Distortion
Nonlinear transformation along the radial direction
New coordinates
Distortion correction: make lines straight
Coordinates of distorted points
Image of a point
Homogeneous coordinates of a 3-D point
Homogeneous coordinates of its 2-D image
Projection of a 3-D point to an image plane
22 Image of a line – homogeneous representation
Homogeneous representation of a 3-D line
Homogeneous representation of its 2-D image
Projection of a 3-D line to an image plane
Image of a line – 2D representations
Representation of a 3-D line
Projection of a line - line in the image plane
Special cases – parallel to the image plane, perpendicular When λ -> infinity - vanishing points In art – 1-point perspective, 2-point perspective, 3-point perspective
23 Orthographic Projection
• Special case of perspective projection – Distance from center of projection to image plane is infinite Image World
– Also called “parallel projection” – What’s the projection matrix?
Slide by Steve Seitz
Perspective distortion
• Problem for architectural photography: converging verticals
Source: F. Durand
24 Perspective distortion
• Problem for architectural photography: converging verticals Shifting the lens Keeping the camera upwards results in a Tilting the camera level, with an ordinary picture of the upwards results in lens, captures only the entire subject converging verticals bottom portion of the building • Solution: view camera (lens shifted w.r.t. film)
http://en.wikipedia.org/wiki/Perspective_correction_lensSource: F. Durand
Perspective distortion
• Problem for architectural photography: converging verticals • Result:
Source: F. Durand
25 Visual Illusions, Wrong Perspective
Vanishing points
Different sets of parallel lines in a plane intersect at vanishing points, vanishing points form a horizon line
26 Ames Room Illusions
More Illusions
Which of the two monsters is bigger ?
27 Approximating an affine camera
Source: Hartley & Zisserman
What is happening here ?
Examples of dolly zoom from movies (YouTube)
28 The dolly zoom
• Continuously adjusting the focal length while the camera moves away from (or towards) the subject • “The Vertigo shot”
Examples of dolly zoom from movies (YouTube)
Real Lens Flaws: Chromatic Aberration • Lens has different refractive indices for different wavelengths: causes color fringing
29 Lens flaws: Spherical aberration
• Spherical lenses don’t focus light perfectly • Rays farther from the optical axis focus closer
Lens flaws: Vignetting • Reduction of brightness at the periphery
30 Radial Distortion – Caused by imperfect lenses – Deviations are most noticeable near the edge of the lens
No distortion Pin cushion Barrel
Radial Distortion
Nonlinear transformation along the radial direction
New coordinates
Distortion correction: make lines straight
Coordinates of distorted points
31 Digital camera
• A digital camera replaces film with a sensor array – Each cell in the array is light-sensitive diode that converts photons to electrons – Two common types • Charge Coupled Device (CCD) • Complementary metal oxide semiconductor (CMOS) – http://electronics.howstuffworks.com/digital-camera.htm
Slide by Steve Seitz
Color sensing in camera: Color filter array Bayer grid Estimate missing components from neighboring values (demosaicing)
Why more green?
Human Luminance Sensitivity Function
Source: Steve Seitz
32