I indirect imaging techniques, e.g., MRI (Fourier), CT (Backprojection) I physical quantities other than intensities are measured I computation leads to 2-D map displayed as intensity Image acquisition Digital images are acquired by I direct digital acquisition (digital still/video cameras), or I scanning material acquired as analog signals (slides, photographs, etc.). I In both cases, the digital sensing element is one of the following: Line array Area array Single sensor Stanley J. Reeves ELEC 7450 - Digital Image Processing Image acquisition Digital images are acquired by I direct digital acquisition (digital still/video cameras), or I scanning material acquired as analog signals (slides, photographs, etc.). I In both cases, the digital sensing element is one of the following: Line array Area array Single sensor I indirect imaging techniques, e.g., MRI (Fourier), CT (Backprojection) I physical quantities other than intensities are measured I computation leads to 2-D map displayed as intensity Stanley J. Reeves ELEC 7450 - Digital Image Processing Single sensor acquisition Stanley J. Reeves ELEC 7450 - Digital Image Processing Linear array acquisition Stanley J. Reeves ELEC 7450 - Digital Image Processing Two types of quantization: I spatial: limited number of pixels I gray-level: limited number of bits to represent intensity at a pixel Array sensor acquisition I Irradiance incident at each photo-site is integrated over time I Resulting array of intensities is moved out of sensor array and into a buffer I Quantized intensities are stored as a grayscale image Stanley J. Reeves ELEC 7450 - Digital Image Processing Array sensor acquisition I Irradiance incident at each photo-site is integrated over time I Resulting array of intensities is moved out of sensor array and into a buffer I Quantized Two types of quantization: intensities are stored as a I spatial: limited number of pixels grayscale image I gray-level: limited number of bits to represent intensity at a pixel Stanley J. Reeves ELEC 7450 - Digital Image Processing Spatial resolution Stanley J. Reeves ELEC 7450 - Digital Image Processing Grayscale resolution Stanley J. Reeves ELEC 7450 - Digital Image Processing Sensors - CCD & CMOS CMOS (complementary CCD (charge-coupled device) metal-oxide-semiconductor) Charge-coupled device. I QE of 19-26%. Whole systems Quantum efficiency of 70% I can be integrated on the same (film has 2% QE). device. Camera-on-chip. Mature technology. In I Standard semiconductor device development since 1969. I manufacturing process. Uses photo-diodes in I Each pixel has read-out conjunction with capacitors to I electronics, amplifiers, noise store charge. correction, and ADC. Charge converted to voltage at I Consume far less power than limited nodes. Varied I CCDs. architectures used for read-out. Need more room for electronics. Most of pixel area is light I I Fill-factor generally not as good sensitive. Good fill-factor. as CCDs. Stanley J. Reeves ELEC 7450 - Digital Image Processing CCD architectures CCDs function in two stages—exposure and read-out I Photons are collected and charge is accumulated during exposure I Area arrays use vertical and horizontal shift registers for read-out I In some architectures, charge is transferred to an inactive/opaque region before readout Linear array Full frame transfer I Pixel intensities are read I The entire pixel area is active sequentially I Time between exposures is significant I Needs mechanical shutter Stanley J. Reeves ELEC 7450 - Digital Image Processing CCD architectures Interline transfer Frame transfer I Charge shifted to adjacent I Need 2x optically active area opaque area and thus are larger and costlier I Subsequently shifted row-wise I Half of the array (for storage) is to a horizontal shift register masked I Complex design (requires I Shutter delay is smaller than full micro-mirrors or microlenses for frame transfer good optical efficiency) Stanley J. Reeves ELEC 7450 - Digital Image Processing Image formation I Both CCD and CMOS sensors are monochromatic I Color images are acquired using color filters overlaid on the sensor The intensity measured at a pixel is Z 1 ci = fi(λ)g(λ)x(λ)l(λ)dλ + ηi −∞ I i = 1;:::; k are distinct color channels sampled at each location I fi(λ) - spectral transmittance of color filter I g(λ) - sensitivity of sensor I x(λ) - spectral reflectance of imaged surface I l(λ) - spectral power density of illuminant I ηi - measurement noise Stanley J. Reeves ELEC 7450 - Digital Image Processing Spectral response of common illuminants Source: http://www.ni.com/white-paper/6901/en/ Stanley J. Reeves ELEC 7450 - Digital Image Processing Multiple sensors I To acquire a 2-D image, multiple CCDs are used to acquire separate color bands Dichroic prism I A dichroic prism is used to split incoming irradiance into narrow-band beams I Red, blue, and green beams directed to separate optical sensors I Issues: cost, weight, registration Beam splitter in action Stanley J. Reeves ELEC 7450 - Digital Image Processing Single sensor acquisition I To avoid the cost and complexity associated with multiple-sensor acquisition, most color digital cameras use a single sensor I Each pixel is overlaid with a color filter such that only one color channel is acquired at a particular pixel location I The Bayer array is the most common color filter array I Green is sampled at twice the density of red and blue since the human visual system (HVS) is more sensitive in the green region of the spectrum I The quincunx sampling arrangement ensures that aliasing in the green channel is least along the horizontal and vertical directions I The full color image is recovered in a post-processing stage known as demosaicking Stanley J. Reeves ELEC 7450 - Digital Image Processing Direct color imaging I The Foveon X3 sensor captures colors at different depths at the same spatial location I The increased density leads to much better spatial resolution I The spectral sensitivity functions at the different layers have substantial overlap I Color separation is a major issue for such sensors Stanley J. Reeves ELEC 7450 - Digital Image Processing Digital camera pipeline Lens assembly Focus control Exposure control I IR blocking (hot I Active auto-focus I Good contrast mirror) systems use IR across image by I Anti-aliasing: emitters to estimate manipulating blurs to increase distance aperture size and spatial I A passive method exposure time correlation dynamically adjusts I Prevents over- among color the focus setting to and channels to help maximimize under-exposed with high-frequency images demosaicking energy Stanley J. Reeves ELEC 7450 - Digital Image Processing Digital camera pipeline I Correct for lens distortion: barrel (fish-eye), pincushion (telephoto), vignetting (reduced brightness at edges) I Gamma correction to compensate for nonlinearity of sensor response (opto-electronic conversion function) I Compensation for dark current. Capture appropriate “dark-image”, subtract from acquired image. I Lens flare (scattered light) compensation (mostly proprietary) Stanley J. Reeves ELEC 7450 - Digital Image Processing Digital camera pipeline I HVS remarkably adaptive; e.g., paper appears white under incandescent light or sunlight I Imaging system will integrate spectral content of irradiance. Without color compensation, images appear unnatural and dissimilar to viewed scenes I White balancing algorithms based on one of two philosophies: I Gray-world assumption R = krR, B = kbB; kr = Gmean=Rmean, kb = Gmean=Bmean I Perfect reflector method Brightest pixel corresponds to white. R = R=Rmax, G = G=Gmax, B = B=Bmax Stanley J. Reeves ELEC 7450 - Digital Image Processing Digital camera pipeline I Reconstruct sparsely sampled signal to form 3-color image I Multitude of methods based on heuristics, properties of the HVS, and mathematical formulations I Since the Bayer array is the most common, most algorithms are tailored Bayer demosaicking specifically for it I Effective algorithms use inter-channel correlation Stanley J. Reeves ELEC 7450 - Digital Image Processing Digital camera pipeline I Captured image is in the digital camera color space. Colors are not impulses at specific wavelength. The sensitivity function of the camera color sensors dictates the camera color space. I The camera-RGB image is transformed to one of many standard color spaces. Most commonly, the transformation is Camera-RGB ! CIEXYZ. I The CIEXYZ space defined by CIE (Commission Internationale de l’Eclairage the International Commission on Illumination) corresponds to the human visual subspace I Many enhancement algorithms use non-RGB color spaces. Stanley J. Reeves ELEC 7450 - Digital Image Processing Digital camera pipeline I Removal of color artifacts due to demosaicking — algorithms based on the constant-hue assumption I Sharpening — performed on luminance component only I Denoising — median filters, bilateral filtering, and thresholding Stanley J. Reeves ELEC 7450 - Digital Image Processing Digital camera pipeline I Display — Images are converted to a format appropriate for display medium (sRGB for monitors, CMY/CMYK for printers). I Compression — Most cameras offer flexible compression options. JPEG is standard in current models. Some JPEG2000. I Storage — Low-end cameras offer only JPEG images as output. Some high-end point-and-shoot cameras and most SLRs will allow for retrieval of RAW images that are unprocessed. RAW images can be processed later on a PC without time and computational constraints. Stanley J. Reeves ELEC 7450 - Digital Image Processing.