<<

Image Formation

Jana Kosecka

3-D Euclidean Space - Vectors

A “free” vector is defined by a pair of points :

Coordinates of the vector :

1 3D Rotation of Points – Euler angles Rotation around the coordinate axes, counter-clockwise:

P ’ ’ Y γ y P X’ x z

Rotation Matrices in 3D

• 3 by 3 matrices • 9 parameters – only three degrees of freedom • Representations – either three Euler angles • or axis and angle representation

• Properties of rotation matrices (constraints between the elements)

2 Rotation Matrices in 3D

• 3 by 3 matrices • 9 parameters – only three degrees of freedom • Representations – either three Euler angles • or axis and angle representation

• Properties of rotation matrices (constraints between the elements)

Columns are orthonormal

Canonical Coordinates for Rotation

Property of R

Taking derivative

Skew symmetric matrix property

By algebra

By solution to ODE

3 3D Rotation (axis & angle)

Solution to the ODE

with

or

Rotation Matrices

Given

How to compute angle and axis

4 3D Translation of Points

Translate by a vector

P ’ t ’ Y x’ x P z’ y z

Rigid Body Motion – Homogeneous Coordinates

3-D coordinates are related by: Homogeneous coordinates:

Homogeneous coordinates are related by:

5 Rigid Body Motion – Homogeneous Coordinates

3-D coordinates are related by: Homogeneous coordinates:

Homogeneous coordinates are related by:

Properties of Rigid Body Motions

Rigid body motion composition

Rigid body motion inverse

Rigid body motion acting on vectors

Vectors are only affected by rotation – 4th homogeneous coordinate is zero

6 Rigid Body Transformation

Coordinates are related by: Camera pose is specified by:

Image Formation

• If the object is our lens the refracted light causes the images

• How to integrate the information from all the rays being reflected from the single point on the surface ?

• Depending in their angle of incidence, some are more refracted then others – refracted rays all meet at the point – basic principles of lenses

• Also light from different surface points may hit the same lens point but they are refracted differently - Kepler’s retinal theory

7 Let’s design a camera

• Idea 1: put a piece of film in front of an object • Do we get a reasonable image?

Slide by Steve Seitz

Pinhole camera

• Add a barrier to block off most of the rays – This reduces blurring – The opening is known as the aperture

Slide by Steve Seitz

8 Pinhole camera model

• Pinhole model: – Captures pencil of rays – all rays through a single point – The point is called Center of Projection (focal point) – The image is formed on the Image Plane

Slide by Steve Seitz

Camera Obscura

• Basic principle known to Mozi (470-390 BCE), Aristotle (384-322 BCE) • Drawing aid for artists: described by Leonardo da Vinci (1452-1519)

Gemma Frisius, 1558

Source: A. Efros

9 Home-made pinhole camera

Why so blurry?

Slide by A. Efros http://www.debevec.org/Pinhole/

Shrinking the aperture

• Why not make the aperture as small as possible? – Less light gets through – Diffraction effects…

Slide by Steve Seitz

10 Shrinking the aperture

Adding a lens

• A lens focuses light onto the film – Thin lens model: • Rays passing through the center are not deviated (pinhole projection model still holds)

Slide by Steve Seitz

11 Adding a lens

• A lens focuses light onto the film – Thin lens model: • Rays passing through the center are not deviated (pinhole projection model still holds) • All parallel rays converge to one point on a plane located at the f

Slide by Steve Seitz

Adding a lens

• A lens focuses light onto the film – There is a specific distance at which objects are “in focus” • other points project to a “circle of confusion” in the image

Slide by Steve Seitz

12 Thin lens formula

Similar triangles everywhere! y’/y = D’/D

D’ D f y y’

image lens object plane Slide by Frédo Durand

Thin lens formula

Similar triangles everywhere! y’/y = D’/D y’/y = (D’-f)/f D’ D f y y’

image lens object plane Slide by Frédo Durand

13 Thin lens formula Any point satisfying the thin lens 1 + 1 = 1 D’ D f equation is in focus.

D’ D f

image lens object plane Slide by Frédo Durand

Depth of Field

• in real camera lenses, there is a range of points which are brought into focus at the same distance • of the lens , as Z gets large – z’ approaches f • human eye – power of accommodation – changing f

14 How can we control the depth of field?

• Changing the aperture size affects depth of field – A smaller aperture increases the range in which the object is approximately in focus – But small aperture reduces amount of light –

need to increase exposure Slide by A. Efros

Varying the aperture

Large aperture = small DOF Small aperture = large DOF Slide by A. Efros

15 Field of View

Slide by A. Efros

Field of View

What does FOV depend on? Slide by A. Efros

16 Field of View

f f

FOV depends on focal length and size of the camera retina

Smaller FOV = larger Focal Length Slide by A. Efros

Field of View / Focal Length

Large FOV, small f Camera close to car

Small FOV, large f Camera far from the car Sources: A. Efros, F. Durand

17 Same effect for faces

wide-angle standard telephoto

Source: F. Durand

Image Formation – Projection

“The School of Athens,” Raphael, 1518

18 Pinhole Camera Model

Pinhole

Frontal pinhole

More on homogeneous coordinates

In homogenous coordinates – these represent the Same point in 3D

The first coordinates can be obtained from the second by division by W

What if W is zero ? Special point – point at infinity – more later

In homogeneous coordinates – there is a difference between point and vector

19 Pinhole Camera Model • Image coordinates are nonlinear function of world coordinates • Relationship between coordinates in the camera frame and sensor plane 2-D coordinates

Homogeneous coordinates

Image Coordinates • Relationship between coordinates in the sensor plane and image

metric coordinates

Linear transformation

pixel coordinates

20 Calibration Matrix and Camera Model • Relationship between coordinates in the camera frame and image Pinhole camera Pixel coordinates

Calibration matrix (intrinsic parameters)

Projection matrix

Camera model

Calibration Matrix and Camera Model • Relationship between coordinates in the world frame and image Pinhole camera Pixel coordinates

More compactly

Transformation between camera coordinate Systems and world coordinate system

21 Radial Distortion

Nonlinear transformation along the radial direction

New coordinates

Distortion correction: make lines straight

Coordinates of distorted points

Image of a point

Homogeneous coordinates of a 3-D point

Homogeneous coordinates of its 2-D image

Projection of a 3-D point to an image plane

22 Image of a line – homogeneous representation

Homogeneous representation of a 3-D line

Homogeneous representation of its 2-D image

Projection of a 3-D line to an image plane

Image of a line – 2D representations

Representation of a 3-D line

Projection of a line - line in the image plane

Special cases – parallel to the image plane, perpendicular When λ -> infinity - vanishing points In art – 1-point perspective, 2-point perspective, 3-point perspective

23 Orthographic Projection

• Special case of perspective projection – Distance from center of projection to image plane is infinite Image World

– Also called “parallel projection” – What’s the projection matrix?

Slide by Steve Seitz

Perspective distortion

• Problem for architectural photography: converging verticals

Source: F. Durand

24 Perspective distortion

• Problem for architectural photography: converging verticals Shifting the lens Keeping the camera upwards results in a Tilting the camera level, with an ordinary picture of the upwards results in lens, captures only the entire subject converging verticals bottom portion of the building • Solution: view camera (lens shifted w.r.t. film)

http://en.wikipedia.org/wiki/Perspective_correction_lensSource: F. Durand

Perspective distortion

• Problem for architectural photography: converging verticals • Result:

Source: F. Durand

25 Visual Illusions, Wrong Perspective

Vanishing points

Different sets of parallel lines in a plane intersect at vanishing points, vanishing points form a horizon line

26 Ames Room Illusions

More Illusions

Which of the two monsters is bigger ?

27 Approximating an affine camera

Source: Hartley & Zisserman

What is happening here ?

Examples of dolly zoom from movies (YouTube)

28 The dolly zoom

• Continuously adjusting the focal length while the camera moves away from (or towards) the subject • “The Vertigo

Examples of dolly zoom from movies (YouTube)

Real Lens Flaws: Chromatic Aberration • Lens has different refractive indices for different wavelengths: causes color fringing

29 Lens flaws: Spherical aberration

• Spherical lenses don’t focus light perfectly • Rays farther from the optical axis focus closer

Lens flaws: Vignetting • Reduction of brightness at the periphery

30 Radial Distortion – Caused by imperfect lenses – Deviations are most noticeable near the edge of the lens

No distortion Pin cushion Barrel

Radial Distortion

Nonlinear transformation along the radial direction

New coordinates

Distortion correction: make lines straight

Coordinates of distorted points

31 Digital camera

• A digital camera replaces film with a sensor array – Each cell in the array is light-sensitive diode that converts photons to electrons – Two common types • Charge Coupled Device (CCD) • Complementary metal oxide semiconductor (CMOS) – http://electronics.howstuffworks.com/digital-camera.htm

Slide by Steve Seitz

Color sensing in camera: Color filter array Bayer grid Estimate missing components from neighboring values (demosaicing)

Why more green?

Human Luminance Sensitivity Function

Source: Steve Seitz

32