COS 429: COMPUTER VISON RADIOMETRY (1 lecture)

• Elements of Radiometry •Radiance • Irradiance •BRDF • Photometric Stereo

Reading: Chapters 4 and 5

Many of the slides in this lecture are courtesy to Prof. J. Ponce Geometry Viewing Lighting Surface albedo

Images I Photometric stereo example

data from: http://www1.cs.columbia.edu/~belhumeur/pub/images/yalefacesB/readme Image Formation: Radiometry

The light source(s)

The sensor characteristics The surface normal

The surface The optics properties

What determines the brightness of an image pixel? The Illumination and Viewing Hemi-sphere

r (θi , φi ) n(x, y) (θo , φo ) At infinitesimal, each point has a tangent plane, and thus a hemisphere Ω.

The ray of light is indexed ρ(x, y) by the polar coordinates (θ , φ) Foreshortening

• Principle: two sources that • Reason: what else can a look the same to a receiver receiver know about a source must have the same effect on but what appears on its input the receiver. hemisphere? (ditto, swapping • Principle: two receivers that receiver and source) look the same to a source • Crucial consequence: a big must receive the same amount source (resp. receiver), of energy. viewed at a glancing angle, • “look the same” means must produce (resp. produce the same input experience) the same effect as hemisphere (or output a small source (resp. receiver) hemisphere) viewed frontally. Measuring Angle

• To define radiance, we require the concept of solid angle • The solid angle sub- tended by an object from a point P is the area of the projection of the object onto the unit sphere centered at P • Measured in steradians, sr • Definition is analogous to projected angle in 2D • If I’m at P, and I look out, solid angle tells me how much of my view is filled with an object Solid Angle of a Small Patch

• Later, it will be important to talk about the solid angle of a small piece of surface dAcosθ dω = r2 A

dω = sinθdθdφ DEFINITION: Angles and Solid Angles

L θ = l = (radians) R

A Ω = a = (steradians) R2 DEFINITION: The radiance is the power traveling at some point in a given direction per unit area perpendicular to this direction, per unit solid angle.

δ2P = L( P, v ) δA δω δ2P = L( P, v ) cosθδA δω PROPERTY: Radiance is constant along straight lines (in vacuum).

δ2P = L( P, v ) δA δω δ2P = L( P, v ) cosθδA δω DEFINITION: Irradiance

The irradiance is the power per unit area incident on a surface.

2 δ P=δE δA=Li( P , vi ) cosθiδωi δΑ

δE= Li( P , vi ) cosθi δωi

E=∫H Li (P, vi) cosθi dωi Photometry

π 2 ⎡ ⎛ d ⎞ ⎤ • L is the radiance. E = ⎢ ⎜ ⎟ cos4 α ⎥ L ⎣⎢ 4 ⎝ z'⎠ ⎦⎥ • E is the irradiance. DEFINITION: The Bidirectional Reflectance Distribution Function (BRDF)

The BRDF is the ratio of the radiance in the outgoing direction to the incident irradiance (sr-1).

Lo ( P, vo ) = ρBD ( P, vi , vo ) δ Ei ( P, vi )

= ρBD ( P, vi , vo ) Li ( P, vi ) cos θi δωi

Helmoltz reciprocity law: ρBD ( P, vi , vo ) = ρBD ( P, vo , vi ) DEFINITION: Radiosity

The radiosity is the total power Leaving a point on a surface per unit area (W * m-2).

B(P) = ∫H Lo ( P , vo ) cosθo dω

Important case: Lo is independent of vo.

B(P) = π Lo(P) DEFINITION: Lambertian (or Matte) Surfaces

A Lambertian surface is a surface whose BRDF is independent of the outgoing direction (and by reciprocity of the incoming direction as well).

ρBD( vi , vo ) = ρBD = constant.

The albedo is ρd = πρBD. DEFINITION: Specular Surfaces as Perfect or Rough Mirrors

θ θ θi θs i s δ

Perfect mirror Rough mirror

Perfect mirror: Lo (P,vs) = Li (P,vi)

n Phong (non-physical model): Lo (P,vo)= ρsLi(P,vi) cos δ

n Hybrid model: Lo (P,vo)= ρd ∫ H Li(P, vi) cosθi dωi+ρsLi(P,vi) cos δ DEFINITION: Point Light Sources

N θi θi S

A point light source is an idealization of an emitting sphere with radius ε at distance R, with ε << R and uniform radiance

Le emitted in every direction. ρ For a Lambertian surface,πε the corresponding radiosity is ⎡ 2 ⎤ N(P)⋅S(P) ≈ ρ (P) B(P) = ⎢ d (P) Le 2 ⎥ cosθi d 2 ⎣ R(P) ⎦ R(P) Local Shading Model

• Assume that the radiosity at a patch is the sum of the radiosities due to light source and sources alone.

No interreflections.

N(P)⋅S (P) • For point sources: B(P) (P) s = ∑ ρd 2 visible s Rs (P)

• For point sources at infinity:

B(P) = ρd (P)N(P) ⋅ ∑Ss (P) visible s Photometric Stereo (Woodham, 1979)

??

Problem: Given n images of an object, taken by a fixed camera under different (known) light sources, reconstruct the object shape. Photometric Stereo: Example (1)

• Assume a Lambertian surface and distant point light sources.

with g(P)=ρ N(P) I(P) = kB(P) = kρ N(P) • S = g(P) •V and V= k S

• Given n images, we obtain n linear equations in g:

T I1 V1.gV1 T I2 V2.gV2 i= … = … = gi= V g g = V-1 i T In Vn.gVn Photometric Stereo: Example (2)

• What about shadows?

• Just skip the equations corresponding to zero-intensity pixels.

• Only works when there is no ambient illumination.

Photometric Stereo: Example (3)

ρ(P) = | g(P) | g(P)=ρ(P)N(P) 1 N(P) = g(P) | g(P) |

Photometric Stereo: Example (3) Integrability! ∂ ⎛ ∂z ⎞ ∂ ⎛ ∂z ⎞ ⎜ ⎟ = ⎜ ⎟ z ∂y ⎝ ∂x ⎠ ∂x ⎝ ∂y ⎠

⎡ ∂z ⎤ ⎢− ⎥ y ∂x ⎧∂z a ⎡a⎤ ⎢ ⎥ = − ⎢ ⎥ ⎢ ∂z ⎥ ⎪∂x c N = b ∝ − ⇒ ⎨ v ⎢ ⎥ ⎢ ∂y ⎥ ∂z b ⎢c⎥ ⎢ ⎥ ⎪ = − ⎣ ⎦ ⎩⎪∂y c u ⎢ 1 ⎥ x ⎣⎢ ⎦⎥

u ∂z v ∂z z(u,v) = (x,0)dx + (u, y)dy ∫0 ∂x ∫0 ∂y

Photometric stereo example

data from: http://www1.cs.columbia.edu/~belhumeur/pub/images/yalefacesB/readme