Characteristics of Digital Fundus Camera Systems Affecting Tonal Resolution in Color Retinal Images 9 ORIGINAL ARTICLE Marshall E Tyler, CRA, FOPS Larry D Hubbard, MAT University of Wisconsin; Madison, WI Wake Forest University Medical Center Boulevard Ken Boydston Winston-Salem, NC 27157-1033 Megavision, Inc.; Santa Barbara, CA [email protected] Anthony J Pugliese Choices for Service in Imaging, Inc. Glen Rock, NJ Characteristics of Digital Fundus Camera Systems Affecting Tonal Resolution in Color Retinal Images INTRODUCTION of exposure, a smaller or larger number of photons strikes his article examines the impact of digital fundus cam- each sensel, depending upon the brightness of the incident era system technology on color retinal image quality, light in the scene. Within the dynamic range of the sen- T specifically as it affects tonal resolution (brightness, sor, it can differentiate specific levels of illumination on a contrast, and color balance). We explore three topics: (1) grayscale rather than just black or white. However, that the nature and characteristics of the silicon-based photo range of detection is not infinite; too dark and no light is sensor, (2) the manner in which the digital fundus camera detected to provide pictorial detail; too bright and the sen- incorporates the sensor as part of a complete system, and sor “maxes out”, thus failing to record detail. (3) some considerations as to how to select a digital cam- This latter point requires further explanation. Sensels era system to satisfy your tonal resolution needs, and how have a fixed capacity to hold electrons generated by inci- to accept delivery and training on a new system.1 dent photons. This capacity is often termed the size of the “electron well”. When the well is empty (no incident DIGITAL PHOTO SENSORS light) there is no rise in sensel voltage. When the well is full (bright incident light), the sensel voltage rises to its In the silicon-based sensor, silicon is the photosensitive maximum and more light has no further effect. By anal- medium. Whereas in film, incident light causes a chemi- ogy, water poured into a well already full to the brim cal change in the silver halide crystals, in the digital sen- will simply spill over the side and thus not increase the sor the incident light stimulates a change in the electron quantity of water contained in the well. Obviously, the state of the silicon. Simply put, photons striking a silicon bigger the capacity of the electronic wells, the broader sensor cause a change in the valence state of this ele- the dynamic range of the image sensor. ment, resulting in a flow of electrons producing a tiny The signal from a sensel is analog, voltage. The amplitude of this electrical and needs to be converted by the sensor current is directly proportional to the circuitry into digital data. Sensors used in number of photons of light striking the fundus cameras typically produce 10-bit sensor, just as in film the amplitude of or 12-bit luminance information, meaning the change in the silver halide crystals that the black/white spectrum is divided is proportional to the quantity of light into 1024 or 4096 steps, respectively. striking the emulsion. How does an array of sensels How is this basic phenomenon har- compose a sensor? nessed to create a workable sensor? If a sensor was composed of a Each sensing pixel (or sensel) making single sensel, it could not capture picto- up the sensor has a base charge, which rial detail, but only the general level of will be interpreted by the camera circuit- ambient illumination. To capture a pic- ry as the black point (no incident light), Figure 1: A charge-coupled device (CCD) ture focused on the sensor (via the cam- and a maximum possible charge, which sensor from a Canon camera – the era optics) requires an array of sensels will be interpreted as the white point active sensor area is the inner rectangle. (Figure 1). The size of this array is (saturated with light). During the period Image courtesy of Steven Merrifield. conventionally expressed in megapixels 10 The Journal of Ophthalmic Photography Volume 31, Number 1 • Spring 2009 (2392x2048 pixels yields 4.8 megapixels), which could HOW A SENSOR CAPTURES COLOR be thought of as the density of the sensels available to The sensel, as described, can only capture a scene in capture spatial detail of the scene being imaged. Sensor monochrome, or shades of grey. Given this inherent limita- resolution is a key variable in calculating the resolving tion, some trickery is required to acquire color. A comple- power of a fundus camera on the retina. ment of color filters (red = R, green = G, and blue = B) is interposed between the incident light source and the sensel TYPES OF SENSOR ARRAYS so that the camera knows which color of light excited that The most common sensor type in current fundus cam- particular sensel. An early approach was to take three iden- eras is the “charge coupled device,” or CCD. This name tical pictures sequentially, varying only color of the filter is based on the mechanism by which the charges of each on the camera. One variant of early color photography sensel in the array are passed to the camera circuitry. on film used the same tactic. Another method still in use During the period of image transmission, the last sensel (and arguably still producing the best quality), is to insert a in each row reads out its charge to the camera, after trichroic prism to divide the incident light into R, G, and B which the second to last sensel passes its charge to the bands, each of which is sent to its own dedicated sensor. last sensel, to which it is thereby “coupled,” and, in turn, That arrangement is often termed a 3-chip camera, after accepts the charge from the third to last sensel, and so the three separate sensors it incorporates. Obviously, both on until the charges from every sensel in the row have of these methods require exquisite alignment in order to been passed sequentially to the camera. While this hap- produce a result in which the color is not smeared. pens very quickly, it cannot happen instantaneously. The The simpler method now in wide-spread use was sequence is controlled by a precise internal clock. patented in 1976 by Bryce E. Bayer of Eastman Kodak The concept leading to the modern digital camera Labs (Rochester, NY), and thus is called the Bayer sensor was published in 1961 by Eugene V. Lally from Jet (i.e., a sensor fitted with a Bayer filter). Rather than put- Propulsion Laboratory (Pasadena, CA) in a paper entitled ting one large filter over each of three sensors (RGB), the “Mosaic Guidance for Interplanetary Travel.”2 This idea Bayer filter superimposes a tiny filter over each individual was implemented into a CCD sensor in 1969 by Willard sensel. The pattern devotes 50% of the sensels to green, Boyle and George E. Smith from AT&T Bell Laboratory. 25% of the sensels to red, and 25% of the sensels to As often happens in technological development, these blue. Green is so favored because that is where most scientists were working on another problem: how to pictorial detail is perceived by human vision. The classic store information in “bubble memory”, similar to today’s Bayer pattern is shown in Figure 2a. flash memory. A CCD can store information as point Without further measures, the color information in voltages that have been passed into it from an external the picture would resemble a French Pointillist painting, source, and then can write that information out again upon demand. Fortuitously, the inven- tors realized that the point volt- ages in the array could be gener- ated directly by incident light if photo-sensitive silicon receptors were incorporated, changing the function of the device from data storage to image acquisition. Consumers of high-end digital cameras (fundus photog- raphers are often among them) a realize that there are other ways of organizing individual sensels into a working array. The com- plementary metallic oxide sensor (CMOS) provides each sensel with its own circuitry to transmit its charge to the camera. While this has advantages (faster image read-out, lower internal “noise”, lower energy draw), at this b point its benefits have not been judged by manufacturers to be Figure 2: Schematic drawings of (a) the Bayer mosaic capture configuration and (b) the Foveon® worth the extra cost. sensor configuration. Drawing courtesy of Foveon, Inc. (San Jose, CA) Characteristics of Digital Fundus Camera Systems Affecting Tonal Resolution in Color Retinal Images 11 composed of individual spots of red, green and blue. To tion entering the eye, and the second the degree of signal attribute R,G,B values to every sensel, the camera soft- amplification by the digital system. The resultant illumina- ware interpolates the missing information for each color tion of the picture is a combination of these two factors. of sensel (e.g., a red sensel lacks its own green and blue The digital fundus camera system truncates the full information) from the surrounding sensels having the range of digital information produced by the sensor in complementary colors. This process is called de-mosaicing. a very important way. To simplify the task of handling One further step smooths out the color even further, and storing digital image information, the camera system although at the cost of exact focus. The incident image collapses the 10-bit or 12-bit data from the sensor (1024 is deliberately defocused slightly in a process called anti- or 4096 grey levels, respectively) into 8-bit images (256 aliasing, to make sure that every red, green, and blue ray grey levels).