Photobit TECHNOLOGY DIGITAL CAMERA-ON-A-CHIP CMOS APS TECHNOLOGY © ER FOSSUM Rev 1 Photobit TECHNOLOGY Imaging Systems Timing & Optics DiDriver Circuits Detector Pixel Array ASSP Analog Signal Processor Analog to Digital Converter ADC Digital Signal Processor DSP © ER FOSSUM Photobit CMOS Image Sensor TECHNOLOGY Camera-on-a-Chip Digital Logic for • User Interface CMOS Active Pixel Color Imaging Array • Sensor Setup • Timing Generator • Digital Signal Processing –Color Processing –White Balance –Image Enhancement • Data Output Formatting AlAnalog SilSignal Processing • Data Sampling • Noise Reduction • Gain Analog-To-Digital Conversion © ER FOSSUM Photobit CMOS Image Sensor TECHNOLOGY Functional Advantages over CCDs •On-chip timing and control circuits •On-chippggpg analog signal processing •Correlated double sampling •Local neighborhood image processing •Multiresolution processing •Compression preprocessing •Pixel defect correction •Ultra high dynamic range •Window readout for electronic pan/tilt •Skip-mode for low resolution readout •On-chip analog-to-digital conversion •High speed digital output •On-chip DSP and digital sensor control •Lower supply voltages and power (5V, 3. 3V, 2. 7V… ) •Easier interface, easier board design © ER FOSSUM Photobit TECHNOLOGY Pixels Vpd Cpd Ipd Vss voltage time © ER FOSSUM Photobit TECHNOLOGY Color Filter Array (CFA) • Each pixel gets covered by a colored filter – Wdbl(RGB)CFAWe use red, green, blue (RGB) CFA - btbest ma thfRGBtch for RGB displays in “Bayer” pattern – Could use complementary colors (cyan, yellow, magenta) 1 090.9 Human Eye 0.8 0.7 0.6 Silicon 0.5 e response vv 0.4 0.3 Relati 0.2 0.1 0 400 500 600 700 800 900 1000 1100 Wavelength (nm) © ER FOSSUM Photobit TECHNOLOGY Fill Factor • A pixel is divided into a sensing portion and a readout portion • Fill factor is the ratio of sensing area to total area and is typically about 20% for CCDs and CMOS APS TtlTotal p ixe l area Sensitive area © ER FOSSUM Photobit TECHNOLOGY Microlenses • Microlenses funnel liggyht away from non-sensitive portions of pixel Microlens layer Color filter layer Metal opaque layer Photodiode Silicon substrate © ER FOSSUM Photobit TECHNOLOGY Pixels • Keyyg to Integrated Ima ggy(ging System (camera on a chi p) is making a good pixel in a CMOS platform process. • Prior to CMOS active pixel was CMOS passive pixel. • Simple and DRAM-like • Not such a good pixel Vpd M1 dthihdidue to high read noise RS Cpd • Gets worse with small Ipd pixels, large arrays and Vss faster readout Vout © ER FOSSUM Photobit TECHNOLOGY Active Pixel • Add an amppplifier inside each pixel. • Not so obvious: – Reduces fill factor due to 3 transistors – Adds fixed pattern noise due to offset variations in each amplifier – Was tried and discarded in the late 1960’ s and then CCDs were invented. • Non-CMOS active ppyixel sensors were demonstrated by Olympus, Toshiba, Texas Instruments and others but suffer from requiring a unique semiconductor fabrication process © ER FOSSUM Photobit TECHNOLOGY Timing Is Everything • CMOS state of the art is ripe for image sensors – Design rules permit competitive pixel sizes – Defects and contamination well controlled – Threshold voltages stable and fairly uniform – AdAnd now foun didries o ffer spec ilidiialized image sensor mo dldules • Customers demand low power, miniaturized systems-on-a-chip • Circuit techniques developed for high performance – Active pixel provides gain in pixel and lower noise – Use of double-correlated sampling and double-delta sampling on- chipppp removes temporal & fixed pattern noise – Column parallel architecture permits low analog bandwidths to reduce noise and artifacts and maintain high frame rate. – Low power imaging circuit techniques reduce power to mW levels © ER FOSSUM Photobit TECHNOLOGY Evolution of Feature Size 100 Image Sensor Pixel Size CCD Invented 10 Practical Size (um) Optical Lim it 1 CMOS Fe ature Size 0.1 1970 1975 1980 1985 1990 1995 2000 2005 Year © ER FOSSUM Photobit TECHNOLOGY Optical Limits 20 18 16 Cheap Lens Resolution (30 lp/mm) 14 F/11 12 10 Airy Disk Diameter (microns) D = 2442.44 λ F# 8 F/2.8 Size 6 4 High Performance Lens Resolution 2 (120 lp/mm) BGRBGR 0 400 500 600 700 800 900 1000 1100 Wavelength (nm) © ER FOSSUM Photobit TECHNOLOGY QUANTUM EFFICIENCY WITH CFA Quantum Efficiency of PB-0100 Rev. C 20 Blue pixels Green pixels Red pixels 15 ficiency, % ficiency, 10 ff 5 uantum E QQ 0 300 400 500 600 700 800 900 1000 1100 WlthWavelength, nm © ER FOSSUM Photobit TECHNOLOGY CMOS Active Pixel • PN junction photodiode (PD) • RST switch M1 charges PD to M1 Vpix Col Vpix RST Bus • Light-generated current Ipd M2 discharges PD SF • Voltage Vpd on PD is gate Vpd voltage of in-pixel source- follower transistor M1. Cpd IdIpd M3 • Row Select (RS) switch M3 RS connects pixel transistor to Vss column bus and charges bus to same voltage as PD. • Voltage Vout picked off at Vout bottom of column. IL © ER FOSSUM Photobit Physical Layout of TECHNOLOGY a CMOS Active Pixel P P P D D D PD DD PHOTODIODE L BUS VV CO P P P D D D M1 RST RESET M2 SF ROW SELECT M3 RS P P P D D D © ER FOSSUM Photobit TECHNOLOGY “Correlated” Double Samppgling Photodiode Voltage integration starts pixel reset level sampled 3V 2V 1V signal sampled 0V 33 ms 66 ms Pixel sampled twice to remove variations in threshold voltaggpe of pixel source follower and reset level. © ER FOSSUM Photobit TECHNOLOGY Simplified Signal Chain PIXEL VDD RST RS GLOBAL COL BUS CS SHS SIG VLP HOR BUS SHR VLP RST VLN PER COLUMN © ER FOSSUM Photobit TECHNOLOGY Electronic Rolling Shutter Read Read Read Reset Reset Reset Row 1 Row 2 Row 3 3V 2V 1V 0V 33 ms 66 ms © ER FOSSUM Photobit X-ray Sensor for Dental and TECHNOLOGY Osteoporosis Applications • Total chip size: 37 x 27 mm • 900 (V) x 675 (H) elements • 40 μm pixel pitch • 2 μm CMOS process • Differential analog output • On-chip timing and control • Perhaps world’s largest CMOS chip in production © ER FOSSUM Xue 1998 Mammography Sensor Photobit TECHNOLOGY (3-Sides Buttable) • 466 x 466 e lemen ts • 3-sides buttable • 30 μm pixel pitch • P-channel PD-type pixel • 0.8 μm process • Differential analog output • 2l2 column loss • Some on-chip circuits Xue 1997 © ER FOSSUM Photobit TECHNOLOGY Pixel Approaches • Passive Pixel Sensor (PPS) – Pho to dio de – Charge injection • Active Pixel Sensor (APS) – Photodiode (dates to 1968) – Photogg(ate (invented at JPL) – Pinned Photodiode (invented by JPL/Kodak) – Floating gate (invented at JPL) – Logarithmic (high dynamic range - low contrast) – Freeze-Frame (Photobit) – More complex amplifiers (from DoD infrared imagers) © ER FOSSUM Photobit TECHNOLOGY Logarithmic Pixel • 3 transistors per pixel • Slightly larger pixel • Great QE VDD • Logarithmic response -- good for large intrascene dynamic range • Poor contrast under flat lig hting • Log-response makes color RS processing extremely complex • Susceptible to FPN • Slow response under low light COL BUS © ER FOSSUM Photobit TECHNOLOGY Pinned Photodiode Active Pixel • 4 transistor pixel VDD • More complex fabrication • Susceptible to lag RST • Great QE • Low noise • Good scaling for large arrays TX • Good scaling for fast arrays RS • First demonstrated by JPL/Kodak PPD COL BUS © ER FOSSUM Photobit TECHNOLOGY Photogate Active Pixel • 5 transistor pixel VDD • Larger pixel • QE loss due to poly gate RST • Lowest noise – 5 e- rms PG TX • GfGood scaling for large arrays RS • Good scaling for fast arrays • US patent no . 5 ,471 ,515 (Pb) COL BUS © ER FOSSUM Photobit TECHNOLOGY On-Chip ADC • Permits full digital interface • Permits on-chip DSP for camera control, color interpolation, etc. • Suppresses noise pick up from EMI, crosstalk • Simplifies interface • Permits faster readout • Can reduce overall chip and/or system power © ER FOSSUM Photobit TECHNOLOGY On-Chip ADC Choices • One (or two or three) • Single-slope ADCs per chip – slow – High data rate • Successive requirement (5-60 approximation Mconv/s) – 8-10 b max – Local high power • Algorithmic dissipation – needs good op-amp • OADCilOne ADC per pixel • Oversampled – Large pixels and low fill factor – Filter takes lots of area • One ADC per column (column-parallel) • Flash and Folding – Tall, skinny ADCs – More power © ER FOSSUM Photobit TECHNOLOGY On-Chip ADC Architectures Digitized in pixel Serial ADC(s) at data rate Column Column parallel parallel ADCs ADCs Multiplexed Parallel output outppput ports © ER FOSSUM Photobit TECHNOLOGY Example: CIF-Resolution Sensor • Internet cameras, games • 360x304 (()p352x288 effective) pixels • 8b flash ADC • 7.9 μm x 7.9 μm (1/5” optical format) • Progg,ressive scan, window readout • 15,000 gate on-chip logic • >30 frames per sec (adj.) • 3.3 V ope r ati on • <100 mW total power • Built-in autoexposure control • On-chip biasing • On-chip CFA • >20 dB SNR @ 1 lux, 30 fps Van Blerkom, Huang, Chan 1999 © ER FOSSUM Photobit TECHNOLOGY Example: VGA-Resolution Sensor • Dual mode cameras, internet cameras • 640x487 (()p640x480 effective) pixels • 320 column-parallel 8b ADCs • 7.9 μm x 7.9 μm (1/3” optical format) • Progg,ressive scan, window readout • 15,000+ gate on-chip logic • >30 frames per sec (adj.) • 5 V ope r ati on • <300 mW total power • Built-in autoexposure control • On-chip biasing • On-chip CFA • >20 dB SNR @ 1 lux, 30 fps Panicacci, Cho 1999 © ER FOSSUM Photobit New Sensors for 2001 TECHNOLOGY • CIF • 7. 8 um pixel • 0.5 um CMOS to 0.35 um CMOS • power reduced 2x • VGA • 7.8 um pixel to 5.6 um pixel • 0.5 um CMOS to 0.35 um CMOS • power reduced 2x © ER FOSSUM Photobit Example: TECHNOLOGY 2Mpixel Digital Still Camera • Digital still cameras • 2 Mpixels • 3.9 um pixel pitch • Microlenses © ER FOSSUM Photobit Major Advantages of CMOS Image TECHNOLOGY Sensors over CCDs • Lower power – Lower supp ly vo ltages (533(5, 3.3, 272.7, ….