Basic Principles of Imaging and Photometry
Lecture #2
Thanks to Shree Nayar, Ravi Ramamoorthi, Pat Hanrahan Computer Vision: Building Machines that See
Lighting Camera
Physical Models
Computer
Scene
Scene Interpretation
We need to understand the Geometric and Radiometric relations between the scene and its image. A Brief History of Images 1558
Camera Obscura, Gemma Frisius, 1558 A Brief History of Images 1558 1568
Lens Based Camera Obscura, 1568 A Brief History of Images 1558 1568
1837
Still Life, Louis Jaques Mande Daguerre, 1837 A Brief History of Images 1558 1568
1837
Silicon Image Detector, 1970
1970 A Brief History of Images 1558 1568
1837
Digital Cameras 1970 1995 Geometric Optics and Image Formation
TOPICS TO BE COVERED :
1) Pinhole and Perspective Projection
2) Image Formation using Lenses
3) Lens related issues Pinhole and the Perspective Projection
Is an image being formed (x,y) on the screen?
YES! But, not a “clear” one.
screen scene
image plane r =(x, y, z) y optical effective focal length, f’ z axis pinhole x
r'=(x', y', f ')
r' r x' x y' y = = = f ' z f ' z f ' z Magnification
A(x, y, z) B y d B(x +δx, y +δy, z) optical f’ z A axis Pinhole A’ d’ x planar scene image plane B’ A'(x', y', f ') B'(x'+δx', y'+δy', f ')
From perspective projection: Magnification:
x' x y' y d' (δx')2 + (δy')2 f ' = = m = = = f ' z f ' z d (δx)2 + (δy)2 z
x'+δx' x +δx y'+δy' y +δy = = Area f ' z f ' z image = m2 Areascene Orthographic Projection
Magnification: x' = m x y' = m y When m = 1, we have orthographic projection
r =(x, y, z)
r'=(x', y', f ') y optical z axis x z ∆ image plane z
This is possible only when z >> ∆z In other words, the range of scene depths is assumed to be much smaller than the average scene depth.
But, how do we produce non-inverted images? Better Approximations to Perspective Projection Problems with Pinholes
• Pinhole size (aperture) must be “very small” to obtain a clear image.
• However, as pinhole size is made smaller, less light is received by image plane.
• If pinhole is comparable to wavelength of incoming light, DIFFRACTION effects blur the image!
• Sharpest image is obtained when:
pinhole diameter d = 2 f ' λ
Example: If f’ = 50mm,
λ = 600nm (red),
d = 0.36mm Image Formation using Lenses
• Lenses are used to avoid problems with pinholes.
• Ideal Lens: Same projection as pinhole but gathers more light!
i o P
P’
f
1 1 1 Gaussian Lens Formula: + = i o f • f is the focal length of the lens – determines the lens’s ability to bend (refract) light
• f different from the effective focal length f’ discussed before! Focus and Defocus
aperture aperture Blur Circle, b diameter d
i o i' o'
1 1 1 1 1 1 f f Gaussian Law: + = + = (i'−i) = (o − o') i o f i' o' f (o'− f ) (o − f )
d Blur Circle Diameter : b = (i ' − i) i'
Depth of Field: Range of object distances over which image is sufficiently well focused. i.e. Range for which blur circle is less than the resolution of the imaging sensor. Two Lens System
d
final object image i2 f2 o2 i1 f1
o1 image plane intermediate virtual image lens 2 lens 1
• Rule : Image formed by first lens is the object for the second lens.
• Main Rays : Ray passing through focus emerges parallel to optical axis. Ray through optical center passes un-deviated.
i i • Magnification: m = 2 1 o2 o1
Exercises: What is the combined focal length of the system? What is the combined focal length if d = 0? Lens related issues
Compound (Thick) Lens Vignetting
L L L B principal planes 3 2 1
α A α nodal points
thickness more light from A than B !
Chromatic Abberation Radial and Tangential Distortion
ideal actual
F F B G ideal FR actual
image plane Lens has different refractive indices for different wavelengths. Radiometry and Image Formation
• To interpret image intensities, we need to understand Radiometric Concepts and Reflectance Properties.
• TOPICS TO BE COVERED:
1) Image Intensities: Overview
2) Radiometric Concepts:
Radiant Intensity Irradiance Radiance BRDF
3) Image Formation using a Lens
4) Radiometric Camera Calibration Image Intensities
source sensor
Need to consider normal light propagation in a cone
surface element
Image intensities = f ( normal, surface reflectance, illumination )
Note: Image intensity understanding is an under-constrained problem! Radiometric concepts – boring…but, important!
source dω (solid angle subtended by dA )
dA' R (foreshortened area) dω θr θi dA (surface area) dA
dA' dA cos θ (1) Solid Angle : ω = = i ( steradian ) (4) Surface Radiance (tricky) : d 2 2 R R 2 d Φ 2 What is the solid angle subtended by a hemisphere? L = (watts / m steradian ) (dA cos θr ) dω dΦ (2) Radiant Intensity of Source : J = ( watts / steradian ) dω • Flux emitted per unit foreshortened area per unit solid angle. Light Flux (power) emitted per unit solid angle
• L depends on direction θr
dΦ (3) Surface Irradiance : E = ( watts / m ) • Surface can radiate into whole hemisphere. dA • L depends on reflectance properties of surface. Light Flux (power) incident per unit surface area.
Does not depend on where the light is coming from! The Fundamental Assumption in Vision
Lighting
No Change in
Radiance
Surface Camera Radiance properties
• Radiance is constant as it propagates along ray – Derived from conservation of flux – Fundamental in Light Transport.
dΦ= L dωω dA = L d dA = d Φ 1 1112 2 2 2
22 dωω12= dA r d 21= dA r
dA dA dωω dA =12 = d dA 11 r 2 2 2
∴=LL12 Relationship between Scene and Image Brightness
• Before light hits the image plane:
Scene Scene Lens Image Radiance L Irradiance E
Linear Mapping!
• After light hits the image plane:
Image Camera Measured Irradiance E Electronics Pixel Values, I
Non-linear Mapping!
Can we go from measured pixel value, I, to scene radiance, L? Relation between Image Irradiance E and Scene Radiance L
image plane surface patch θ dAs dωs α α dωi image patch dωL dAi
f z
• Solid angles of the double cone (orange and green): 2 dAi cos α dAs cos θ dA cos α z dω = dω = s = i s α 2 α 2 ( f / cos ) (z / cos ) dAi cos θ f • Solid angle subtended by lens: (1) π d 2 cosα dω = (2) L 4 (z / cosα)2 Relation between Image Irradiance E and Scene Radiance L image plane surface patch θ dAs dωs α α dωi image patch dωL dAi
f z
• Flux received by lens from dA s = Flux projected onto image dA i
(3) L (dAs cos θ ) dωL = E dAi
2 π d 4 • From (1), (2), and (3): E = L cos α 4 f
• Image irradiance is proportional to Scene Radiance!
• Small field of view Effects of 4th power of cosine are small. Relation between Pixel Values I and Image Irradiance E
Image Camera Measured Irradiance E Electronics Pixel Values, I
• The camera response function relates image irradiance at the image plane to the measured pixel intensity values. g : E → I
(Grossberg and Nayar) Radiometric Calibration •Important preprocessing step for many vision and graphics algorithms such as photometric stereo, invariants, de-weathering, inverse rendering, image based rendering, etc.
g −1 : I → E
•Use a color chart with precisely known reflectances.
255
g −1 ?
g Pixel Values
0 0 ? 1 90% 59.1% 36.2% 19.8% 9.0% 3.1% Irradiance = const * Reflectance
• Use more camera exposures to fill up the curve. • Method assumes constant lighting on all patches and works best when source is far away (example sunlight).
• Unique inverse exists because g is monotonic and smooth for all cameras. The Problem of Dynamic Range
• Dynamic Range: Range of brightness values measurable with a camera
(Hood 1986)
• Today’s Cameras: Limited Dynamic Range
High Exposure Image Low Exposure Image
• We need 5-10 million values to store all brightnesses around us. • But, typical 8-bit cameras provide only 256 values!! High Dynamic Range Imaging
• Capture a lot of images with different exposure settings.
• Apply radiometric calibration to each camera.
• Combine the calibrated images (for example, using averaging weighted by exposures).
(Mitsunaga)
(Debevec)
Images taken with a fish-eye lens of the sky show the wide range of brightnesses.