566 IEEE SENSORS JOURNAL, VOL. 2, NO. 6, DECEMBER 2002 Polarization Imaging: Principles and Integrated Polarimeters Andreas G. Andreou and Zaven Kevork Kalayjian Abstract—Polarization is a general descriptor of light and con- of Huygens, Young, Fresnel, and Maxwell to lead us to the tains information about reflecting objects that traditional inten- understanding of the polarization of light that we have today; sity-based sensors ignore. Difficult computer vision tasks such as and, its study has lead to a deeper understanding of light and image segmentation and object orientation are made tractable with polarization vision techniques. Specularities, occluding contours, light-matter interaction. and material properties can be readily extracted if the Stokes po- Polarization is used to classify chemical isomers [22], to an- larization parameters are available. Astrophysicists employ polar- alyze solar and atmospheric phenomena [23], [24], to investi- ization information to measure the spatial distribution of magnetic gate stress and strain through photoelasticity, to classify mate- fields on the surface of the sun. In the medical field, analysis of the rials and recognize objects [25]–[30], and to analyze reflections polarization allows the diagnose of disease in the eyes. The retinae of most insect and certain vertebrate species are sensitive to polar- [31], [32]. Polarimetric devices and techniques have been in- ization in their environment, but humans are blind to this prop- vented to filter, separate, quantify, and modulate polarized light. erty of light. Biologists use polarimeters to investigate behaviors of Polaroid filters are used in magneto-optic drives, liquid crystal animals—vis-à-vis polarization—in their natural habitats. In this displays, and 3-D movie glasses. Polarization is used to track paper, we first present the basics of polarization sensing and then eye movements [33], and polarization microscopes help biolo- discuss integrated polarization imaging sensors developed in our laboratory. gists see things that would otherwise escape human perception [34]. Polarization measuring instruments today are either quasi- I. INTRODUCTION static and/or do not involve an imaging device (i.e. an array of ISUALIZING the polarization of light is beyond the ca- photosensitive sites with sensitivity to the state of polarization). V pabilities of the human eye, but it is an everyday behavior The rotating polarization analyzer type polarization imaging de- for many other creatures of the Earth. Bees use the polariza- vice, for example a liquid crystal polarization camera [35], [36] tion of scattered sunlight as a compass [1], [2]. Waterstriders, can give dynamic information in the millisecond time scale. backswimmers, and dragonflies use the polarization of reflected However, in the time between measurements the brightness can light to detect water and also under-water predators [3]–[7]. change, even if slightly, making it difficulty if not impossible to Both invertebrates (e.g. mantis shrimp [8], octopi [9], crayfish disentangle brightness variations from polarization information. [10], beetles [11], flies [12]) and vertebrates (e.g. rainbow trout For real-time, portable polarization imaging, an additional [13], [14], salmon [15], sunfish [16], [17], salamanders [18], and factor is the complexity of the data processing equipment and lizards [19]) are able to sense and use polarization in their envi- their size. We have fabricated and tested a series of experi- ronments. One species of polarization sensitive cephalopod, the mental prototypes for integrated CMOS polarization imagers cuttlefish, can modulate the reflectivity of its skin, thereby ma- [37]–[40]. In this paper we present a tutorial introduction to po- nipulating the reflected pattern of polarization for camouflage or larization phenomena and discuss a number of integrated sen- during copulation—like a polarization communication channel sory devices that have been engineered for polarization imaging [20]. As a biological visual modality, polarization is prevalent [41]. among animals. Although unable to naturally sense polarization, we humans II. POLARIZATION OF LIGHT:THE BASICS have still been able to measure and analyze polarization in our environment. In the history of science, the first recorded obser- Let us consider an electromagnetic wave propagating along vation of a polarization related phenomenon was made by the the z-axis, with the e-vectors confined to oscillate along only Danish professor Erasmus Bartolinus in 1669: the double-re- one perpendicular axis, either or . fraction of light through an Iceland spar [21]. It took the likes (1) Manuscript received September 11, 2002; revised December 27, 2002. This work was supported by DARPA/ONR MURI N00014-95-1-0409 with Boston where and are the magnitude of the and components University on Automated Sensing and Vision Systems and by National Science Foundation grant EIA-0130812. The associate editor coordinating the review of of the electric field, and is the phase difference between these this paper and approving it for publication was Prof. R. W. Newcomb. two components. The appearance of in the above equa- A. G. Andreou is with the Department of Electrical and Computer Engi- tion allows for the most general case where orthogonal compo- neering, Johns Hopkins University, Baltimore MD 21218 USA. Z. K. Kalayjian is with Epigy Labs, Yerevan, Armenia. nents have different phase that may depend on spatial or tem- Digital Object Identifier 10.1109/JSEN.2003.807946 poral coordinates. Substitute for in each of the 1530-437X/02$17.00 © 2002 IEEE ANDREOU AND KALAYJIAN: POLARIZATION IMAGING: PRINCIPLES AND INTEGRATED POLARIMETERS 567 of the orthogonal components and their phase difference, the Stokes vector becomes (5) In order to fully describe the state of polarization of light, three (a) (b) (c) quantities must be known: , , and the phase difference Fig. 1. Polarization depends on the mode of oscillation of the electric field. (a) Circularly polarized, (b) linearly polarized, and (c) unpolarized. III. HOW DOES LIGHT BECOME POLARIZED? components in the equation and temporarily drop the explicit The origins of polarized radiation can be separated into two expression of the spatial and temporal dependence of in (1). categories: 1) radiation that is polarized upon emission from a polarized (2) light source; (3) 2) radiation that is originally unpolarized, but upon which the quality of polarization is subsequently imparted. By tracing the location of the tip of the electric field vector as It is interesting to note that the vast majority of polarized elec- the wave moves through space, we see that the resulting loci tromagnetic sources from category #1 are human made: lasers, ˇ describe a helix. synchrotron radiation, Cerenkov radiation, radio and television The ellipticity and orientation of the ellipse depend on the pa- broadcasts, and the Zeeman effect, to name some. Collision po- rameters . When the electric field oscillates in this larization in solar flares is one of the rare cases of a naturally mode, it is said to be elliptically polarized. In fact, if occurring polarized light source [24]. These phenomena share a , light is always elliptically polarized. The special case common trait—asymmetry or anisotropy (from a dipole, mag- where , and makes the ellipse into a netic field vector, or velocity vector) that results in a preferred circle, and the radiation is said to be circularly polarized. direction of oscillation of emitted radiation [43]. Another special case happens when and the Polarized radiation from category #2 above also depends on ellipse degenerates into a line. This is called linear polarization. an asymmetry. Unpolarized radiation, as was mentioned, ex- Depending on how the energy of the electromagnetic radiation is hibits no long-term preference for a particular oscillatory mode. distributed between the components , the electric field Consequently, in order for it to become polarized, some prefer- can be linearly polarized parallel to or . There is yet another ence has to be applied, via asymmetric work, to the unpolarized possible generally categorizable oscillatory mode that occurs radiation. This can be achieved through scattering, reflection, when and are not deterministically related. In other words, birefringence, or dichroism; that latter two will be described in there is no consistent temporal relationship between the phases more detail below as they relate to the integrated polarization of the orthogonal electric fields. In this case, the e-vector takes a sensors discussed in this paper. random path as the wave travels through space. This radiation is A. Birefringence called natural or unpolarized. Fig. 1 shows the “head-on” view for circularly, linearly, and unpolarized light. In an isotropic medium, the electric field and the displace- All the possible states of polarization can be represented in ment vector are related by the scalar , the permittivity of the one vector known as the Stokes vector [42], named after George material. However, in an anisotropic medium, permittivity is a Gabriel Stokes, who introduced it in 1852. The Stokes vector tensor quantity is a shorthand notation that describes the polarization of light through four components (6) The result of this tensor relationship in anisotropic crystals is (4) that and are not always parallel, which means that the di- rection of energy flow, or the ray direction, is not parallel to the wavefront normal, or the wave-normal