Lecture Notes 10 Image Sensor Optics
• Imaging optics ◦ Space-invariant model ◦ Space-varying model • Pixel optics ◦ Transmission ◦ Vignetting • Microlens
EE 392B: Image Sensor Optics 10-1 Image Sensor Optics
Microlens
Imaging Optics
Pixel Optics
Image sensor optics consist of (a) imaging optics and (b) pixel optics
EE 392B: Image Sensor Optics 10-2 Imaging Optics: Linear Space-Invariant Model
Y Y
Entrance Exit Pupil Pupil X X
Z
Object Generalized Model Image L (W/m2 sr) (f/#, T, R) I (W/m2)
A generalized model for rotationally symmetric imaging optics
J.W. Goodman, Introduction to Fourier Optics, 2nd Ed, Ch 6, pp. 126-171 (1996)
EE 392B: Image Sensor Optics 10-3 Imaging Optics: Camera Equation
T (λ )R(x, y;λ ) ( λ )≅ π x y λ Iideal x, y; Lscene , ; 1+ 4(1+ m)2 ( f /#)2 m m
2 Iideal : ideal, unaberrated, geometric image irradiance distribution (W m )
2 Lscene: Lambertian object radiance distribution (W sr m )
= − m zi zo : magnification T : transmittance R : relative illumination factor (e.g., cos4 fall-off) f /# = f D: f-number
Imaging optics map 2D object radiance distribution (W/sr m2) into 2D image irradiance distribution (W/m2)
M.V. Klein and T.E. Furtak, Optics, , 2nd Ed, Ch 4, pp. 193-262 (1986)
EE 392B: Image Sensor Optics 10-4 Resolution Limited by Diffraction
Plane Converging Plane Converging wave wave wave wave
Perfect Diffraction geometric limited point point
Infinite Aperture Optics Finite Aperture Optics
Diffraction, which is caused at aperture edges, is the fundamental reason why a plane wave can not be focussed into a perfect geometric point
EE 392B: Image Sensor Optics 10-5 Diffraction: Space Domain Description
Image formation process with incoherent illumination can be described in the space domain by a convolution operation
( λ)= ( λ)∗ ( λ) Iimage x, y; PSF x, y; Iideal x, y;
Iimage: blurred, aberrated, distorted image irradiance distribution
Iideal : ideal, unaberrated, geometric image irradiance distribution
PSF: point spread function, i.e., response of optical system in image plane to a point excitation in object plane ∗: convolution operator
EE 392B: Image Sensor Optics 10-6 Diffraction: Frequency Domain Description
Alternatively, image formation in shift-invariant systems can be viewed as a linear filtering process in the frequency domain ( λ) = ( λ) ⋅ ( λ) F {Iimage x, y; } F {PSF x, y; } F {Iideal x, y; } = λ ⋅ ( λ) OTF(fx , f y ; ) F {Iideal x, y; }
( λ)= −1 λ ⋅ ( λ) Iimage x, y; F {OTF(f x , f y ; ) F {Iideal x, y; }}
The optical transfer function (OTF) is normalized to preserve radiometry. Its magnitude is the modulation transfer function (MTF)
F {PSF(x, y;λ)} λ = λ = MTF(fx , f y ; ) OTF(fx , f y ; ) F {PSF(x, y;λ)} = = fx 0, f y 0
EE 392B: Image Sensor Optics 10-7 Example: Diffraction-limited Lens
1 f/# = 2.0, λ = 486 nm 0.9
0.8
0.7
0.6
0.5 MTF 0.4
0.3
0.2
0.1
0 0 200 400 600 800 1000 1200 Spatial frequency (lp/mm)
2 − −1 1 MTF(fr) = π(ϕ cos ϕ sin ϕ) with ϕ = cos (frλf/#) and fr,cutoff = λf/#
EE 392B: Image Sensor Optics 10-8 System MTF
If the system is linear and space-invariant, the system MTF (optics + sensor) in the frequency domain can then be easily computed
λ = λ ⋅ ⋅ λ MTFsystem (f x , f y ; ) MTFoptics (fx , f y ; ) MTFgeometric (fx , f y ) MTFdiffusion (fx , f y ; )
To obtain the image spectrum we apply the MTF as a linear filter
λ = λ ⋅ λ Iimage (fx , f y ; ) MTFsystem (fx , f y ; ) Iobject (fx , f y ; )
EE 392B: Image Sensor Optics 10-9 Modeling Real Imaging Lenses
Vignetting Distortion Illumination Effects
Wavefront Point Spread Modulation Transfer Aberrations Function (PSF) Function (MTF)
P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 48-58 (2005)
EE 392B: Image Sensor Optics 10-10 Imaging Optics: Linear Space-Varying Model
Y Y Ω Entrance Exit k Pupil Pupil X X
Z
Object Generalized Model Image L (W/m2 sr) (f/#, T, R) I (W/m2)
Image plane is segmented into isoplanatic regions: Ω1, Ω2, . . ., Ωn
P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 48-58 (2005)
EE 392B: Image Sensor Optics 10-11 Linear Space-Varying Image Formation
Let Ωk, k = 1, . . . , n be aplanatic image segments, then
I (x, y;λ)= ∑PSFΩ (x, y;λ)∗ I Ω (x, y;λ) image k ideal, k k or −1 I (x, y;λ)= F ∑OTFΩ (f , f ;λ)⋅ F {I Ω (x, y;λ)} image k x y ideal, k k
P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 48-58 (2005)
EE 392B: Image Sensor Optics 10-12 Example: Double Gauss f/2.0 Lens
P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 48-58 (2005)
EE 392B: Image Sensor Optics 10-13 Double Gauss f/2.0 Lens: MTF
P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 48-58 (2005)
EE 392B: Image Sensor Optics 10-14 Pixel Optics
Incident Photons
Reflected Transmitted Photons Photons iciency f Rejected or Accepted Scattered Photons Optical Ef Photons
Uncollected Collected Electrons
iciency Electrons f Quantum Ef
P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002) H. Rhodes et al., IEEE Workshop on Microelectronics and Electron Devices, pp. 7-18 (2004)
EE 392B: Image Sensor Optics 10-15 Incident Photons
-3 x 10 10
9 c) 8 x-se u 7 e (l ur
s 6 po
x 4000
E 5
c i tr
e 4
3 hotom 2000 P 2 1000 1 2 3 4 5 6 7 8 Pixel size (µm)
P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 1-13 (2005)
EE 392B: Image Sensor Optics 10-16 Pixel Transmittance
F+(z’) F-(z’)
Layer 0 I01 Layer 1 L1
Layer j-1 Transmission Ij-1,j Layer j Lj
Layer m Lm Im,m+1 Layer m+1
F+(z”) F-(z)
F. Abeles, Ann. de Phys. 5, pp. 596-640 (1950) P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)
EE 392B: Image Sensor Optics 10-17 Example: Transmittance 0.18µm CMOS
1
0.8
0.6
0.4 air Transmittance 0.7 µm Si3N4
0.2 8.3 µm SiO2
Si 0 400 500 600 700 800 Wavelength (nm)
Pixel transmittance is λ-dependent (even for dispersion-free materials)
P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)
EE 392B: Image Sensor Optics 10-18 Example: Transmittance 0.18µm CMOS
0.8
0.6
Si3N4 0.4 nsmission T a r SiO2 T
0.2 Si
0 0 20 40 60 80 100 Angle (degrees)
Pixel transmittance (λ-averaged) is approximately independent of angle
P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)
EE 392B: Image Sensor Optics 10-19 Pixel Vignetting
No Vignetting
Pixel Vignetting Pixel Cross-talk
P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)
EE 392B: Image Sensor Optics 10-20 Pixel Vignetting: Effect of Pixel Height
0.7 0 degrees 11 degrees 0.6 23 degrees
0.5
0.4
0.3 Pixel response 0.2
0.1
0 1 2 3 4 5 6 7 Number of metal layers
Reduction in optical efficiency as a function of the number of metal layers in a 0.18µm standard CMOS process (f/1.8 imaging lens)
P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)
EE 392B: Image Sensor Optics 10-21 Pixel Vignetting: Effect of Technology Scaling
0.8 0 degrees 11 degrees 0.7 23 degrees se 0.6 pon res l e
x 0.5 i P
0.4
0.3 0.4 0.35 0.3 0.25 0.2 0.15 Feature size (µm)
Reduction in optical efficiency for a standard APS pixel with a 30% fill-factor using 2 metal layers as a function of the feature size of CMOS technology
P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002)
EE 392B: Image Sensor Optics 10-22 Optical Efficiency
• Definition: ◦ Optical efficiency (OE) is the ratio of the photons incident of the substrate and the photons incident on the pixel surface • Sources of photon loss:
◦ Back-reflections in dielectric stack (air-SiO2, SiO2-Si) ◦ Photons absorbed in dielectric stack (SiON) ⇒ Pixel transmittance T(λ,θ) ◦ Photons scattered away from pixel (pixel cross-talk) ◦ Photons rejected by metal ⇒ Pixel vignetting V(x,y,θ) • Description: ◦ OE(x,y,λ,θ) = T(λ,θ) V(x,y,θ)
EE 392B: Image Sensor Optics 10-23 Microlens
• Focus light onto photo-sensitive region – increases effective fill factor from 25-40% to 60-80% (and sensitivity by ≥ 2X) • Less effective if photosensitive area is irregularly shaped
A. Theuwissen, “Solid State Imaging with Charge-Coupled Devices,” Kluwer (1995) S.G. Wuu et al., “High performance 0.25 µm CMOS color imager technology with non-silicide source/drain pixel,” IEDM Tech. Dig., pp. 705-708 (2000)
EE 392B: Image Sensor Optics 10-24 Microlens Fabrication
• Lens material requirements: ◦ Highly transparent in the visible light region ◦ Index of refraction > 1.59 ◦ Can be applied below 500C ◦ No degradation or aging ◦ Semiconductor processing com- patible ◦ Can be patterned with feature size commensurate with the pixel size • Lens materials are typically i-line or DUV resists • Base materials are acrylic-based resists, polyimide resists, epoxy resists, poly- organosiloxane, polyorganosilicate
EE 392B: Image Sensor Optics 10-25 Microlens and the Main Lens
• Microlens is optimized for a specific main lens system • Rays incident on the microlens form a cone with NA = sin ϕ • NA varies as a function of the size and position of the exit pupil • Principle ray at the periphery of the sensor has an angle δ, chief ray angle (CRA), between the ray and the optical axis (δ depends on the position of the pixel on the sensor)
A. Theuwissen, “Solid State Imaging with Charge-Coupled Devices,” Kluwer (1995)
EE 392B: Image Sensor Optics 10-26 Microlens and F-Number
• High F-number: rays are parallel (NA ≈ 0) • Low F-number: rays arrive at an angle (NA large) – microlens effectiveness low
A. Theuwissen, “Solid State Imaging with Charge-Coupled Devices,” Kluwer (1995)
EE 392B: Image Sensor Optics 10-27 Micro lens: How to concentrate photons
Micro-lens
Metal Wires
Photodiode
Shallow Pixel Deep Pixel
EE 392B: Image Sensor Optics 10-28 Micro lens: How to concentrate photons
Dµ-lens Dµ-lens
fµ-lens f'µ-lens
Shallow Pixel Deep Pixel
EE 392B: Image Sensor Optics 10-29 Micro lens: How to concentrate photons
2NA 2NA'
Shallow Pixel Deep Pixel
NA = 1 NA0 = 1 q1+4(f/#)2 q1+4(f/#0)2
EE 392B: Image Sensor Optics 10-30 Micro lens: How to concentrate photons
2.44 f/# λ
2.44 f/#' λ
Shallow Pixel Deep Pixel
0 f f f/# = µLens f/#0 = µLens DµLens DµLens
EE 392B: Image Sensor Optics 10-31 Micro lens: How to concentrate photons
• Conservation of etendue
⇒ G = 2NAimagingwpixel = 2NAµlenswdiode
• Etendue limits light collection efficiency (NA) ⇒ Bigger NA allows bigger etendue • Concentration depends on ratio of NA of microlens and imaging lens
⇒ For a 2x space concentration (wdiode = wpixel/2) ⇒ 2NAµlens > NAimaging
• Diffraction limits spot size (f-number) ⇒ Smaller f-number allows smaller spot size
EE 392B: Image Sensor Optics 10-32 Micro lens: Concentration
(a) (b) f/4.0 f/2.8 10 f/2.0 64 f/4.0 8 36 f/2.8 on ) i t m a r µ t
f/2.0 ( n 16 t e h 6 f/1.4 c g n ei o h
C 4 f/1.4 xel
Pi 4 0 10 8 10 6 8 6 2 4 4 2 2 2 4 6 8 10 Pixel size (µm) Pixel height (µm) Pixel size (µm)
P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 1-13 (2005)
EE 392B: Image Sensor Optics 10-33 Pixel Position
• For large CRA (pixel away from the center of the lens, or exit pupil close to the sensor), ray may not focus on the photo-sensitive region • Effect: non-uniform sensitivity profile across image sensor
A. Theuwissen, “Solid State Imaging with Charge-Coupled Devices,” Kluwer (1995)
EE 392B: Image Sensor Optics 10-34 Micro lens: Redirection (w/o offset)
Pixel width: 3µm Pixel Height: 8 µm 1 f/1.4 f/2.0 f/2.8 0.8 f/4.0
0.6 OE
0.4
0.2
0 0 5 10 15 20 θ (deg) CR P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 1-13 (2005)
EE 392B: Image Sensor Optics 10-35 Micro lens: Redirection (with offset)
Pixel width: 3µm Pixel Height: 8 µm
1 f/4.0 f/2.8 0.8
f/2.0 0.6
f/1.4 0.4
∆ = fml tan(θ) Optical Efficiency
0.2
fml θ 0 0 5 10 15 20 θ (deg)
P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 1-13 (2005)
EE 392B: Image Sensor Optics 10-36 Optical Efficiency: Summary
• Without microlens: ◦ OE(x,y,λ,θ) = T(λ,θ) V(x,y,θ) • With microlens: ◦ OE(x,y,λ,θ) = T(λ,θ) V(x,y,θ) ML(x,y,θ) ∗ where ML(x,y,θ) represents a correction factor to account for the concentration and/or redirection performed by the microlens
EE 392B: Image Sensor Optics 10-37 Improving OE
• Optimizing dielectric stack thickness • Make sure dielectrics are not light absorbing • Utilize different dielectric materials to achieve total internal reflection • Add an airgap between pixels (total internal reflection)
EE 392B: Image Sensor Optics 10-38