CMOS Evolution: THE DGITAL ON A CHIP GETS ACTVE Active Pixel CMOS imaging tech, developed at the Jet Propulsion lab, is now commercialized into awaited "camera-on-a-chip" apps. The inside story. By Dr. Eric R. Fossum Figure J. Photobit generators and signal processing chips to J/4" format complete the system. These various ancillary 5 J2x.384 element chips add to the component cost and manu­ digital color cam­ facturing cost of a CCD system. In a CMOS era-on-a-chip with APS camera system, these functions are all pixel array, analog signal processor integrated onto a single chip, keeping com­ (ASP), analog-to­ ponent and manufacturing costs low. digital converter A third significant advantage of the and on-chip inter­ CMOS APS camera-on-a-chip is reduced face, timing, control power consumption. Because the entire sys­ and smart function tem is integrated on a single chip, the power digital block. required for inter-chip I/O in a CCD system is eliminated. Another source of power con­ techniques. This is sumption in a CCD system is the large and because the camera is varied voltage levels required by CCD dri­ basically a single chip ving circuitry. Each power supply in a CCD and a lens, although system results in power inefficiency, and other digital signal pro­ since power is voltage X current (IV), or cessing (DSP) chips capacitance X voltage X voltage X frequency might be added to the (CV2t), the use of higher voltages in a CCD system to provide for system has a direct impact on power dissipa­ compression and inter­ tion. Low voltage CMOS (e.g. 3.3V) technol­ face functions. Minia­ ogy can reduce power. Typically, a complete turization is important CMOS APS camera-on-a-chip with digital for many portable output dissipates under 75 mW. he -on-a-chip, based on applications such as digital still , lap­ There are other advantages associated CMOS active pixel sensor (APS) tech­ top computers, PDAs and automobiles, as well with the CMOS APS technology. A key nology, is now entering the marketplace. as for unobtrusive cameras in desktop video­ advantage is functionality. Because the APS I The digital carnera-on-a-chip has a full dig­ conferencing and other fixed-camera uses. technology allows for nearly random access ital interlace so that there are no analog sig­ A second major advantage is cost. While to each pixel, many functions previously diffi­ nals going into the chip or coming out of it. CCDs and CMOS image might be cult to implement with CCDs can be readily Supply voltage biases other than VDD (e.g. 3.3 priced competitively (especially now that perlormed with the APS technology, such as V) are generated on-chip or through simple CMOS image sensors resistor dividers. The chip has a serial inter­ are competing on the face for setting up it's operational mode and market), a CMOSAPS Photodetector reading out control status registers internal to camera system can be the chip. Output pixel data is digital with much less than a CCD embedded digital line and frame sync signals. camera system. This is This system-on-a-chip is a major advance because CCD camera for camera system designers for many rea­ systems require many Active amplifier sons. First, the camera can be easily miniatur­ supporting compo­ Row ized without resorting to exotic packaging nents such as timing Select

Figure 2. Active Dr. Eric R. Fossum led the development of CMOS pixel sensors have APS technology at JPL, and is now Chief Scientist an amplifier built with Photobit Corp. He is also Adjunct Professor into every pixel for Column of at UCLA. For more info lower noise and Output Photobit can be reached at 818-248-4393. faster readout. NOVEMBER 1997 13 readout of smal1 windows of interest for generation deep space exploration spacecraft. puter or sister video processing chips. This machine vision and tracking applications, Such imaging systems are driven by perfor­ approach permits adaptive processing to meet electronic pan and zoom for consumer appli­ mance, not cost. This latter effort was led by a wide variety of applications, but requires a cations and ultra high intrascene dynamic our group at the U.S. Jet Propulsion Laborato­ sister chip or computer. It is a particularly range imaging. ry with subsequent transfer of the technology important approach when interface bandwidth to AT&T Bel1 Labs (Lucent), Kodak, Intel, is limited (e.g. USB or Firewire) since video Background: How did we get here? National Semiconductor and several other chips produce a large amount of data and color There are several important forces that major U.S. companies, and the startup of Pho­ interpolation triples that data volume. have brought the CMOS APS camera-on-a­ tobit. The convergence of the efforts, com­ At the June 1997 IEEE Workshop on chip to the forefront of advanced imaging bined with customer demand for miniatur­ CCDs and Advanced Image Sensors in attention. MO~type image sensors had their ized, low-power and cost-effective imaging Brugge, Belgium, the Jet Propulsion Labora­ genesis in the late 1960's. Both passive and systems, has led to significant advances in tory described a very low power 256x256 ele­ active pixels were examined, but the perfor­ CMOS image sensors and the development ment sensor designed for NASA space appli­ mance of the early MOS technology was poor of the CMOS APS camera-on-a-chip. cations with a serial interface for both com­ and consequently image quality of the sen­ The CMOS APS achieves high imaging mand and data output, on-chip DACs for sors was concomitantly poor and unreliable. performance for two reasons. First, unlike the programmable internal biases (e.g. ADC ref­ When the CCD was invented in late 1969, passive pixel, the active pixel provides gain erence) and self-calibrating ADCs. At the its charge domain operation solved many of within the pixel. Charge that is generated by August 1997 Hot Chips Symposium at Stan­ pixel illumination is ford, Photobit announced its optical format amplified during read­ 512x384 digital color camera-on-a-chip. This out and used by ana­ sensor delivers 8b raw data at 3-39 ful1 log signal processing frame/s (programmable) using on-chip ana­ (ASP) circuits at the log to digital converters and features a serial bottom of each col­ interface port for controlling window of inter­ umn. Second, unlike est size and position, on-chip autoexposure its early MOS prede­ logic and low power operation. cessors, our approach The second camp advocates putting as to CMOS APS uses much of the color interpolation and other correlated double sam­ digital signal processing on the same chip as pling (CDS) to achieve the . This is important for a vari­ higher quality images. ety of stand-alone applications and for direct The use of CDS has replacement of CCD-based digital camera two important advan­ systems. For example, at the 1997 Interna­ tages. It reduces tem­ tional Solid-State Circuits Conference poral noise from the (ISSCC) in San Francisco, a paper by Mat­ Figure 3. Photobit's 10 cm2 dental X-ray sensor is in volume production for a custom-design customer. (Radiograph cour­ pixel readout circuitry, sushita described a research project where a tesy of Schick Technologies) and it suppresses fixed low resolution sensor was integrated with pattern noise (FPN) ADC and DCT compression circuitry. At the the operational problems of MOS image sen­ from the in-pixel amplifiers. It was this combi­ 1998 ISSCC, Lucent Technologies will pre­ sors. Since then, CCDs have achieved extra­ nation of active pixel plus column-paral1el sent a paper on an experimental 352x288 ordinarily high levels of performance with ASP circuits that led to the first high perfor­ CMOS APS with on-chip digital color interpo­ low readout noise, high dynamic range and mance CMOS APS, demonstrated by our lation operating at 30 frame/so At the same excel1ent responsivity. Like the marvel of group at JPL in 1993. meeting, VLSI Vision will present a 306x244 Boeing 747 flight, the CCD is highly complex Contributing to the recent activity in single-chip color camera with analog com­ and its functionality is a testament to nearly CMOS image sensors is the steady, exponen­ posite color video NTSC output encoding. 30 years of superb engineering evolution. tial improvement in CMOS technology. The In addition to mainstream market applica­ Any other imaging technology, such as rate of minimum feature size decrease has tions, there are many niche applications for CMOS APS, must achieve similar levels of outpaced similar improvements in CCD tech­ CMOS imaging technology. Photobit offers a performance to displace CCD technology, or nology. Furthermore, sensor pixel size is lim­ custom image sensor design service for par­ have other overwhelming advantages such ited by both optical physics and optics cost, ticular applications that can benefit from a as very low power or new functions. making moot the CCD's inherent pixel size tailored CMOS APS design, and has deliv­ advantage for most applications. Recent ered numerous sensors to high profile cus­ CMOS out of the JPL progress in on-chip signal processing (and tomers such as Kodak, Lucent and lIT. In the early 1990's, two independently off-chip DSP) has also reduced CMOS image One interesting application area is in med­ motivated efforts led to a resurgence in sensor FPN to acceptable levels. In addition, ical X-rays. At the 1998 ISSCC, Photobit will CMOS image sensor development. The first the transition from analog imaging and dis­ present a paper on a CMOS dental X-ray effort was to create highly functional single­ play systems to digital cameras tethered to chip it developed with a strategic partner chip imaging systems where low cost, not PCs permits digital FPN correction with neg­ over the past few years. This "camera-on-a­ performance, was the driving factor. This ligible system impact. chip" is probably one of the world's largest effort was spearheaded by separate commercial CMOS chips, measuring over researchers at the University of Edinburgh Two camps now 37mm x 28mm (>10 cm2). The chip, placed in Scotland (later becoming the firm At the present time there are two philosoph­ in the patient's mouth, detects the onset of VISION) and Linkoping University in Swe­ ical camps in digital cameras-on-a-chip, X-ray irradiation, integrates the X-ray image, den Oater becoming IVP) . depending on the application of the chip. In and self-initiates the readout of the pixel data The second independent effort grew from one camp (advocates include Intel, Kodak and and a subsequent dark reference frame. NASA's need for highly miniaturized, low­ Photobit) color interpolation and image com­ (See picture above) power, instrument imaging systems for next­ pression takes place off the chip in a host com­ (continued on page 74) 14 ADVANCED IMAGING with up to four hosts and up to eight disk media or an image processing system. P50 has a new crystal pigment tube that drive channels. Active/Active Redundant Also, an RS-232C port permits remote offers better contrast, more uniform color Controller support provides failover protec­ control of camero functions. Apps include reproduction and lower power consump­ tiori for mission-critical apps, while the machine vision, surveillance, archiving, tion than shadow-mask tubes The P50 remote Dial-In capability allows system medical imaging, etc. PULNIX has digital on-screen controls, a .27mm setup, diagnostics and maintenance proce· CIRCLE 215 dot pitch, up to 1280 x 1024 res and dures to be performed from remote loca­ is TCO 92-compliant. PANASONIC tions. Half and full-height systems in tower or C++ GRAPHICAL IMAGING COMPUTER PERIPHERAL COMPANY rackmount configurations features environ­ Image ++ is an obiect oriented graphical CIRCLE 218 mental monitoring and alert systems, redun­ development tool for C++ image process­ dant hot swappable power supplies and ing applications running under Windows LINEAR CCD W/ANALOG fans and automatic remote fault-notification. 95 or Windows NT. The package allows The XRD4417 is a linear CCD digitizing Supports RAID Levels 0, 1, 0+ 1, 3 & 5 con­ the user to build up a block diagram of sub-system for flat-bed and sheet-fed currently. Systems are scalable to 19+ Ter­ their image processing algorithm and see scanners, as well as multifunction print­ abytes CYBERSTORAGE SYSTEMS, INC the results. Functions can then be modified ers and CCD imagers. It's CCD inter­ CIRCLE 213 if required at C++ source code level to face circuitry requires only an external fine tune the algorithm. Microsoft Visual decoupling capacitor to perform corre­ PROGRAMMABLE FRAME GRABBER C++ and Borland C++ are supported lated double sampling (CDS) of on input The HI*DEF Accura is a programmable with source code editing and compilation. signal. A 6-bit programmable gain high speed frame grabber that captures A library of over 50 image processing amplifier (PGA) in the device increases video from any non-standard video source functions are also provided in the pack­ the signal to the appropriate level to and provides digital images for mission age. KANE COMPUTING match the 10-bit A/D converter input critical applications. The HI * DEF Accura CIRCLE 216 range. The XRD4417 samples at 1.54 offers four soft­ million samples per second and offers ware selec­ ill-RES MICRO CAMERA low-noise and operates from a single 5V table inputs to Recently introduced is a micro-camera sys­ power supply. EXAR CORPORATION capture high­ tem for microscopy, achieving a 1.4 Mil­ CIRCLE 219 speed analog lion pixel resolution through a special signals in real­ color CCD hi-res digital camera with a New product listings are provided as an informational time with pro­ 2/3" Vacancy Transfer imager. This sys­ service for our readers. They are not advertisements grammable tem is a low-cost solution to a variety of or product endorsements. Inclusion is entirely at the discretion of Advanced Imaging editors. analog sam­ microscopy apps. Various lenses are pling rates up available in addition to a C-mount to 140 MHz adapter, allowing for stand-alone use or with less than in conjunction with a conventional micro­ FOSSUM ±500 pi cosec­ scope. A digital printer is available that (continued/rom page 14) , onds of jiller. produces near photo quality hard-copy -=-:0...... _ The Accura and image archiving software brings What's next? can be connected to four different analog images into the PC for processing, storage The history of microelectronics teaches sources such as outputs from CT, MR, and network output SCALAR AMERICA us that integration leads to greater reliabili­ Ultrasound, and Nuclear Medicine CIRCLE 217 ty, lower system power and plummeting devices. Plus, it also captures color images system cost/performance ratios. CMOS­ from non-interlaced computer displays up ON-SCREEN MENU CRTS based imaging systems-on-a-chip can be to 1280x1024@75Hz IMAGRAPH The 17" P70 is an upgraded version of expected to reflect these long-standing CIRCLE 214 the P-17 with improved horizontal fre­ trends as they develop over the next five quency and additional controls for rota­ years. We can expect the emergence of a 1.3K X 1K PROGRESSIVE SCAN tion adjustment and color temperature. It plethora of imaging applications and the The TM-1 300 is a 1O-bit camera that fea­ insertion of image capture systems in many tures a 1300(H) x 1030(Vj, 2/3" format aspects of our personal lives. Look for low­ interline type progressive scan CCD with cost single-chip cameras in your car, office, the capability to asynchronously capture home and doctor's office. Widespread video images of objects in motion. The TM­ teleconferencing is inevitable as are debates over privacy versus security. The great battle between CCDs and CMOS APS is just beginning as CMOS APS begins to not only open new markets but to absorb market segments previously "owned" by CCDs. CCD manufacturers can be expect­ ed to counter with cost reductions (as has already begun) and with lower power features an ultra-fine 0.25mm dot pitch, requirements. While CCD cameras could, in 1300 scans at a 12 frame/sec. rate and 16" diagonal viewing area and 1600 x principle, be shrunk to two-chip solutions features an internal converter which out­ 1280 max resolution. The P70 is (CCD plus a do-all CMOS chip), improved puts a real-time 45fps signal for display TC092-compliant and incorporates DO­ functional capabilities and low power advan­ on an SXGA monitor. An internal 8-bit DAF electronic gun technology which tages are fundamental to CMOS-based imag­ frame store captures and holds randomly produces 20 percent less spot aberra­ ing technology. The next five years will be triggered images for playout to storage tion than conventional guns. The 15" interesting times, indeed. • 74 ADVANCED IMAGING