Sensors Simplified ObjecBves

• Cameras’ Sensors (and Film) where Rays of • Photographic Processes for Digital Capture Light become Pixels • The Five Simplified layers of a CCD • Two benefits of using the Camera Raw

Reading: hHps://en.wikipedia.org/wiki/Charge-coupled_device

Digital: Converting Light to Data Digital: Converting Light to DataDigital: ConverBng Light to Data Digital: ConverBng Light to Data • Micro lenses: Capture Light and direct it • CCD: Charge-Coupled CCD: Charge-Coupledto the light sensiBve areas of the Device, a device CCDDevice, a device for : Charge-Coupled Device, asensor. device converBng electrical for converting electrical– Added Lens layer charge, into a forcharge, into a digital value converting electrical charge,• Hot Mirror into: Simple Filtering. Lets visible a • Pixelsdigital are represented by value light pass, but reflects lights in the digitalcapacitors, which convert value invisible part of the spectrum (depends on kind of light to capture –e.g., and store (accumulates) infrared, ultraviolet- UV camera). Pixelsincoming photons as Pixelsare represented are represented by capacitors,Also provides AnB-aliasing. by capacitors, electron charges • Color Filter: Bayer Array separate light which• Willard Boyle and George whichconvert convert and store and incoming storeinto RGB. photons incoming photons E. Smith, 1969 (Won a • Photo Diodes: Color Blind. But measure as Noble Prize in Physics in electronas electron charges charges intensiBes, and energy converted to 2009). electrons Willard Boyle and George E.• Deple8on LayerSmith, 1969: Where Electron Are Willard Boyle and Georgecollected (part of photo diodes layer) E. Smith, 1969 Smith and hHps://Boyleen.wikipedia.org/wiki/Charge-coupled_device Smith and Boyle (Won a Noble Prize in Physics in 2009). commons.wikimedia.org (Won a Noble Prize in Physics in 2009). © 2015 Irfan Essa, Georgia Tech, All Rights Reserved commons.wikimedia.org © 2015 Irfan Essa, Georgia Tech, All Rights Reserved

Digital: ConverBng Light to Data Dual Microlens Designs Shrinks Pixel Size Sony CCD • Microlens Array: Capture Light and direct it to the light sensiBve areas of the sensor. – Added Lens layer Hot Mirror Bayer Filter

• Cuts IR light. • Bayer Filter on a • Hot filter, that Filters out Digital: Converting Light to Data sensor infrared light to miBgate sensors that are sensiBve • Pixel : to the infrareds (750nm – square of 4 color 1250), and contaminated squares. colors. • Lets only 1 color – E.g., Fires light through. • Cold Filter – Cold Mirror filters IR into the pahtway – 3 PaHerns one for (more common infront of Combine each color lens. Bayer filter on a sensor – CombinaBon gives us the final image

Images: Wikipedia © 2015 Irfan Essa, Georgia Tech, All Rights Reserved

Digital: Convert Light to Data Converting Light to Data Convert Bayer to RGB

• R = 200 • G = 170, 180 ? • B = 153

Raw input in Bayer mosaic format RGB color planes

© 2015 Irfan Essa, Georgia Tech, All Rights Reserved

Convert Bayer to RGB Bayer Filter

• R = 200 • G = AVG(170, 180)=175 • B = 153 Demosaic ½ Green • Combine Bayer Filter & Interpola8on to esBmate other colors from the With this method sensor 1/4 Blue incomplete sample by only captures Raw color Bayer Filter Mechansism data for 1/3 of the image. Use Demosaic in order 1/4 Red ch03.qxd 12/4/04 02:42 PM Page 36 to esBmate the • Bayer Filter remaining 2/3 of the

image. hHps://en.wikipedia.org/wiki/Demosaicing

cross over small amounts of other colors beyond those in their chromic make-up. hHp://www.csee.wvu.edu/~xinl/papers/demosaicing_survey.pdf This allows crossover through the green filter meaning that energies from the red and blue spectral areas will affect the green sites on the sensor. The second reason for having more green filters than red or blue is that blue is the noisiest channel and restricting its use produces a cleaner image. Another way to capture the color is via the Foveon X3® chip. This sensor uses a natural attribute of silicon to separate the colors. Depending on wavelength, a color will penetrate to varying depths into silicon, similar to what occurs with color film. Figure 3-5 Arrangement of filters for This factor, along with the stacking of three layers of sites, allows the sensor to capture photocells for a matrix or Bayer array. the light’s color at each stack of photoreceptors. While theoretically there should be no crossover, there is some absorption of colors as the different wavelengths travel through succeeding sensitivity layers. Sensors are also manufactured that do not have color capture filters or do not acquire color. These cameras are designed to produce monochromatic, black and white images or are usedOther Filters with tricolor photography. Using a monochro- matic sensor to produce tricolor photography, filters are placed in the light path to

Figure 3-6 and Figure 3-7 The Foveon X3® chip stacks photodiodes to capture all light at a single location. The sensitive areas vary as shown in the colorized patent drawing.

36 sensors • Foveon: stacks photodiodes to capture all light at a single locaBon. • Fujifilm Super CCD (more neighbors) à Claims more data to work with for interpolaBon hHp://www.vividlight.com/arBcles/2514.htm

Camera RAW File Format

• Contains minimally processed data from the sensor • Image encoded in a device-dependent colorspace • Captures radiometric characterisBcs of the scene • Viewable image from the camera's sensor data • Like a photographic negaBve – Has a wider dynamic range or color; preserves most of the informaBon of the captured image