Multi-Agent Recognition System Based on Object Based Image Analysis Using Worldview-2

Total Page:16

File Type:pdf, Size:1020Kb

Multi-Agent Recognition System Based on Object Based Image Analysis Using Worldview-2 intervals, between −3 and +3, for a total of 19 different EVs To mimic airborne imagery collection from a terrestrial captured (see Table 1). WB variable images were captured location, we attempted to approximate an equivalent path utilizing the camera presets listed in Table 1 at the EVs of 0, radiance to match a range of altitudes (750 to 1500 m) above −1 and +1, for a total of 27 images. Variable light metering ground level (AGL), that we commonly use as platform alti- tests were conducted using the “center weighted” light meter tudes for our simulated damage assessment research. Optical setting, at the EVs listed in Table 1, for a total of seven images. depth increases horizontally along with the zenith angle, from The variable ISO tests were carried out at the EVs of −1, 0, and a factor of one at zenith angle 0°, to around 40° at zenith angle +1, using the ISO values listed in Table 1, for a total of 18 im- 90° (Allen, 1973), and vertically, with a zenith angle of 0°, the ages. The variable aperture tests were conducted using 19 dif- magnitude of an object at the top of the atmosphere decreases ferent apertures ranging from f/2 to f/16, utilizing a different by 0.28 at sea level, 0.24 at 500 m, and 0.21 at 1,000 m above camera system from all the other tests, with ISO = 50, f = 105 sea level, to 0 at around 100 km (Green, 1992). With 25 mm, four different degrees of onboard vignetting control, and percent of atmospheric scattering due to atmosphere located EVs of −⅓, ⅔, 0, and +⅔, for a total of 228 images. between sea level and 1,000 m, this would mean a horizontal Two DSLR camera systems were used in this study, a Nikon image at 1,000 m would have 25 percent less path radiance D800E full frame camera paired with a Nikon DC-Nikkor 105 than the same image captured at sea level. Given both the mm f/2 to f/16 relative aperture lens, which was used for the non-linear and temporally varying nature of optical depth, a variable aperture study, and a Sony Alpha 65 APS-C camera 25 percent reduction in path radiance was an estimate used (Bockaert, 2006) paired with a Sony f = 18-55 mm f/3.5 to 5.6 for this study. We simulated vertical imaging altitudes of 750 relative aperture lens, which was used for the variable EV, WB, m and 1,500 m AGL, such that the 565 m to 1,125 m in the light meter, and ISO studies. The Sony a65, shown setup for horizontal plane, above ground level. our experimental imaging in Figure 3, would have been the In addition to images datasets generated for the change only camera utilized for this study, however the widest aper- detection noise minimization tests (described above), an- ture usable to capture the 55 mm images, f/5.6, is an aperture other set was created that simulated infrastructure damage. at which vignetting effects are negligible. For this reason, Simulated cracks were used to test the sensitivity of different the Nikon D800E with a wider f/2 aperture lens with more capture settings in detecting cracks in a wall or road as prox- pronounced vignetting effects was utilized for the aperture ies for infrastructure damage. Image pairs taken of simulated testing (DXOMARK, 2012). cracks simulated were analyzed to determine the settings for which the smallest crack can be resolved from difference Table 1. Experimental camera exposure variables and their images and to examine the influence of EV settings on crack settings. detection. To assess the ability to detect damage signals, simu- EVs White Light Metering Relative lated cracks made out of black painter’s tape were placed on (19) Balance (9) EVs (7) ISO (6) aperture (19) a concrete wall with widths approximately equal to one, two, three, and four pixels (GSD = 4.5 cm). Vertical cracks were cre- -3 AWB -3 100 f/2 ated with varying widths, equal to one, two, three, and four -2 ⅔ 2500K -2 200 f/2.2 times the estimated image ground sampling distance (GSD), -2 ⅓ 3500K -1 400 f/2.5 and diagonal cracks were created equal to one, two, and four -2 4500K 0 800 f/2.8 times the estimated GSD, using matte black tape. An oblique -1 5500K +1 1600 f/3.2 imagery collection was performed with the Sony camera mounted on a tripod from the top of building 42 m vertically, 0 6500K +2 3200 f/3.5 960 m horizontally distant from the concrete wall. A second +1 7500K +3 f/9 collection (repetition of EV settings) made after the tape had +2 8500K f/10 been removed. Image pairs for each EV setting were regis- +2 ⅓ 9500K f/11 tered and then subset to the extent covering the simulated cracks and their surrounding background (concrete wall). The +2 ⅔ f/13 images for this test were collected with only a single WB of +3 f/14 AWB, compared to the nine WB settings captured for the noise f/16 minimization test imagery, all other settings utilizing the con- trolled constants from the noise minimization image set. The The Nikon D800E was used to test across-image variability ideal EV level is that which enables detection of the smallest during a single collection, utilizing a reflectance calibration simulated cracks, with the greatest temporal brightness differ- panel as a surface of uniform brightness. The reflectance ence across the simulated crack features. panel was positioned perpendicular to the sun, with the camera positioned as close to perpendicular with the sun and reflectance panel as possible, without having the shadow of Data Analysis the camera in the resulting image. Images were captured in Automated change detection products are typically based on the mid-afternoon, to lessen the chances of accidental overex- image differencing of one or more wavebands or transform im- posure when capturing images with a high f-stop. ages for a RSI pair (in our case), creating an output a raster with The Sony camera captured four image sets for two different band values representing the magnitude and sign of DN value scenes in support of dynamic range and spatial acuity tests changes. The most appropriate (if not optimal) imagery collec- with variable EV, WB, light metering, and ISO. Two collections tion settings were those combination of settings which produce were carried out in the afternoon with a solar elevation of images with the lowest root mean square difference (RMSD) val- 45°, and two collections were carried out in the late afternoon ues for DNs in the difference images, for test images or subsets with a solar elevation of 30°, to achieve variations in illumi- where scenes contained few actual land surface changes. nation conditions and degrees of shadowing. Images were Prior to image analyses, test images were downloaded, captured on clear and hazy days, and yielding image data sets sorted, and assessed for quality, for both the noise minimiza- influenced by two differing atmospheric optical conditions. tion and simulated crack signal maximization assessments. The sites chosen were selected to capture differing urban Test image pairs used for dynamic range and light metering, scene compositions, and to achieve an image viewing azimuth vignetting, and simulated crack signal assessments, required angle parallel to the solar azimuth at time of collection. 152 March 2018 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING.
Recommended publications
  • Breaking Down the “Cosine Fourth Power Law”
    Breaking Down The “Cosine Fourth Power Law” By Ronian Siew, inopticalsolutions.com Why are the corners of the field of view in the image captured by a camera lens usually darker than the center? For one thing, camera lenses by design often introduce “vignetting” into the image, which is the deliberate clipping of rays at the corners of the field of view in order to cut away excessive lens aberrations. But, it is also known that corner areas in an image can get dark even without vignetting, due in part to the so-called “cosine fourth power law.” 1 According to this “law,” when a lens projects the image of a uniform source onto a screen, in the absence of vignetting, the illumination flux density (i.e., the optical power per unit area) across the screen from the center to the edge varies according to the fourth power of the cosine of the angle between the optic axis and the oblique ray striking the screen. Actually, optical designers know this “law” does not apply generally to all lens conditions.2 – 10 Fundamental principles of optical radiative flux transfer in lens systems allow one to tune the illumination distribution across the image by varying lens design characteristics. In this article, we take a tour into the fascinating physics governing the illumination of images in lens systems. Relative Illumination In Lens Systems In lens design, one characterizes the illumination distribution across the screen where the image resides in terms of a quantity known as the lens’ relative illumination — the ratio of the irradiance (i.e., the power per unit area) at any off-axis position of the image to the irradiance at the center of the image.
    [Show full text]
  • Sample Manuscript Showing Specifications and Style
    Information capacity: a measure of potential image quality of a digital camera Frédéric Cao 1, Frédéric Guichard, Hervé Hornung DxO Labs, 3 rue Nationale, 92100 Boulogne Billancourt, FRANCE ABSTRACT The aim of the paper is to define an objective measurement for evaluating the performance of a digital camera. The challenge is to mix different flaws involving geometry (as distortion or lateral chromatic aberrations), light (as luminance and color shading), or statistical phenomena (as noise). We introduce the concept of information capacity that accounts for all the main defects than can be observed in digital images, and that can be due either to the optics or to the sensor. The information capacity describes the potential of the camera to produce good images. In particular, digital processing can correct some flaws (like distortion). Our definition of information takes possible correction into account and the fact that processing can neither retrieve lost information nor create some. This paper extends some of our previous work where the information capacity was only defined for RAW sensors. The concept is extended for cameras with optical defects as distortion, lateral and longitudinal chromatic aberration or lens shading. Keywords: digital photography, image quality evaluation, optical aberration, information capacity, camera performance database 1. INTRODUCTION The evaluation of a digital camera is a key factor for customers, whether they are vendors or final customers. It relies on many different factors as the presence or not of some functionalities, ergonomic, price, or image quality. Each separate criterion is itself quite complex to evaluate, and depends on many different factors. The case of image quality is a good illustration of this topic.
    [Show full text]
  • Section 10 Vignetting Vignetting the Stop Determines Determines the Stop the Size of the Bundle of Rays That Propagates On-Axis an the System for Through Object
    10-1 I and Instrumentation Design Optical OPTI-502 © Copyright 2019 John E. Greivenkamp E. John 2019 © Copyright Section 10 Vignetting 10-2 I and Instrumentation Design Optical OPTI-502 Vignetting Greivenkamp E. John 2019 © Copyright On-Axis The stop determines the size of Ray Aperture the bundle of rays that propagates Bundle through the system for an on-axis object. As the object height increases, z one of the other apertures in the system (such as a lens clear aperture) may limit part or all of Stop the bundle of rays. This is known as vignetting. Vignetted Off-Axis Ray Rays Bundle z Stop Aperture 10-3 I and Instrumentation Design Optical OPTI-502 Ray Bundle – On-Axis Greivenkamp E. John 2018 © Copyright The ray bundle for an on-axis object is a rotationally-symmetric spindle made up of sections of right circular cones. Each cone section is defined by the pupil and the object or image point in that optical space. The individual cone sections match up at the surfaces and elements. Stop Pupil y y y z y = 0 At any z, the cross section of the bundle is circular, and the radius of the bundle is the marginal ray value. The ray bundle is centered on the optical axis. 10-4 I and Instrumentation Design Optical OPTI-502 Ray Bundle – Off Axis Greivenkamp E. John 2019 © Copyright For an off-axis object point, the ray bundle skews, and is comprised of sections of skew circular cones which are still defined by the pupil and object or image point in that optical space.
    [Show full text]
  • Holographic Optics for Thin and Lightweight Virtual Reality
    Holographic Optics for Thin and Lightweight Virtual Reality ANDREW MAIMONE, Facebook Reality Labs JUNREN WANG, Facebook Reality Labs Fig. 1. Left: Photo of full color holographic display in benchtop form factor. Center: Prototype VR display in sunglasses-like form factor with display thickness of 8.9 mm. Driving electronics and light sources are external. Right: Photo of content displayed on prototype in center image. Car scenes by komba/Shutterstock. We present a class of display designs combining holographic optics, direc- small text near the limit of human visual acuity. This use case also tional backlighting, laser illumination, and polarization-based optical folding brings VR out of the home and in to work and public spaces where to achieve thin, lightweight, and high performance near-eye displays for socially acceptable sunglasses and eyeglasses form factors prevail. virtual reality. Several design alternatives are proposed, compared, and ex- VR has made good progress in the past few years, and entirely perimentally validated as prototypes. Using only thin, flat films as optical self-contained head-worn systems are now commercially available. components, we demonstrate VR displays with thicknesses of less than 9 However, current headsets still have box-like form factors and pro- mm, fields of view of over 90◦ horizontally, and form factors approach- ing sunglasses. In a benchtop form factor, we also demonstrate a full color vide only a fraction of the resolution of the human eye. Emerging display using wavelength-multiplexed holographic lenses that uses laser optical design techniques, such as polarization-based optical folding, illumination to provide a large gamut and highly saturated color.
    [Show full text]
  • Aperture Efficiency and Wide Field-Of-View Optical Systems Mark R
    Aperture Efficiency and Wide Field-of-View Optical Systems Mark R. Ackermann, Sandia National Laboratories Rex R. Kiziah, USAF Academy John T. McGraw and Peter C. Zimmer, J.T. McGraw & Associates Abstract Wide field-of-view optical systems are currently finding significant use for applications ranging from exoplanet search to space situational awareness. Systems ranging from small camera lenses to the 8.4-meter Large Synoptic Survey Telescope are designed to image large areas of the sky with increased search rate and scientific utility. An interesting issue with wide-field systems is the known compromises in aperture efficiency. They either use only a fraction of the available aperture or have optical elements with diameters larger than the optical aperture of the system. In either case, the complete aperture of the largest optical component is not fully utilized for any given field point within an image. System costs are driven by optical diameter (not aperture), focal length, optical complexity, and field-of-view. It is important to understand the optical design trade space and how cost, performance, and physical characteristics are influenced by various observing requirements. This paper examines the aperture efficiency of refracting and reflecting systems with one, two and three mirrors. Copyright © 2018 Advanced Maui Optical and Space Surveillance Technologies Conference (AMOS) – www.amostech.com Introduction Optical systems exhibit an apparent falloff of image intensity from the center to edge of the image field as seen in Figure 1. This phenomenon, known as vignetting, results when the entrance pupil is viewed from an off-axis point in either object or image space.
    [Show full text]
  • Optics II: Practical Photographic Lenses CS 178, Spring 2013
    Optics II: practical photographic lenses CS 178, Spring 2013 Begun 4/11/13, finished 4/16/13. Marc Levoy Computer Science Department Stanford University Outline ✦ why study lenses? ✦ thin lenses • graphical constructions, algebraic formulae ✦ thick lenses • center of perspective, 3D perspective transformations ✦ depth of field ✦ aberrations & distortion ✦ vignetting, glare, and other lens artifacts ✦ diffraction and lens quality ✦ special lenses • 2 telephoto, zoom © Marc Levoy Lens aberrations ✦ chromatic aberrations ✦ Seidel aberrations, a.k.a. 3rd order aberrations • arise because we use spherical lenses instead of hyperbolic • can be modeled by adding 3rd order terms to Taylor series ⎛ φ 3 φ 5 φ 7 ⎞ sin φ ≈ φ − + − + ... ⎝⎜ 3! 5! 7! ⎠⎟ • oblique aberrations • field curvature • distortion 3 © Marc Levoy Dispersion (wikipedia) ✦ index of refraction varies with wavelength • higher dispersion means more variation • amount of variation depends on material • index is typically higher for blue than red • so blue light bends more 4 © Marc Levoy red and blue have Chromatic aberration the same focal length (wikipedia) ✦ dispersion causes focal length to vary with wavelength • for convex lens, blue focal length is shorter ✦ correct using achromatic doublet • strong positive lens + weak negative lens = weak positive compound lens • by adjusting dispersions, can correct at two wavelengths 5 © Marc Levoy The chromatic aberrations (Smith) ✦ longitudinal (axial) chromatic aberration • different colors focus at different depths • appears everywhere
    [Show full text]
  • Projection Crystallography and Micrography with Digital Cameras Andrew Davidhazy
    Rochester Institute of Technology RIT Scholar Works Articles 2004 Projection crystallography and micrography with digital cameras Andrew Davidhazy Follow this and additional works at: http://scholarworks.rit.edu/article Recommended Citation Davidhazy, Andrew, "Projection crystallography and micrography with digital cameras" (2004). Accessed from http://scholarworks.rit.edu/article/468 This Technical Report is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in Articles by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected]. Projection Crystallography and Micrography with Digital Cameras Andrew Davidhazy Imaging and Photographic Technology Department School of Photographic Arts and Sciences Rochester Institute of Technology During a lunchroom conversation many years ago a good friend of mine, by the name of Nile Root, and I observed that a microscope is very similar to a photographic enlarger and we proceeded to introduce this concept to our classes. Eventually a student of mine by the name of Alan Pomplun wrote and article about the technique of using an enlarger as the basis for photographing crystals that exhibited birefringence. The reasoning was that in some applications a low magnification of crystalline subjects typically examined in crystallograohy was acceptable and that the light beam in an enlarger could be easily polarized above a sample placed in the negative stage. A second polarizer placed below the lens would allow the colorful patterns that some crystalline structures achieve to be come visible. His article, that was published in Industrial Photography magazine in 1988, can be retrieved here. With the advent of digital cameras the process of using an enlarger as an improvised low-power transmission microscope merits to be revisited as there are many benefits to be derived from using the digital process for the capture of images under such a "microscope" but there are also potential pitfalls to be aware of.
    [Show full text]
  • Part 4: Filed and Aperture, Stop, Pupil, Vignetting Herbert Gross
    Optical Engineering Part 4: Filed and aperture, stop, pupil, vignetting Herbert Gross Summer term 2020 www.iap.uni-jena.de 2 Contents . Definition of field and aperture . Numerical aperture and F-number . Stop and pupil . Vignetting Optical system stop . Pupil stop defines: 1. chief ray angle w 2. aperture cone angle u . The chief ray gives the center line of the oblique ray cone of an off-axis object point . The coma rays limit the off-axis ray cone . The marginal rays limit the axial ray cone stop pupil coma ray marginal ray y field angle image w u object aperture angle y' chief ray 4 Definition of Aperture and Field . Imaging on axis: circular / rotational symmetry Only spherical aberration and chromatical aberrations . Finite field size, object point off-axis: - chief ray as reference - skew ray bundels: y y' coma and distortion y p y'p - Vignetting, cone of ray bundle not circular O' field of marginal/rim symmetric view ray RAP' chief ray - to distinguish: u w' u' w tangential and sagittal chief ray plane O entrance object exit image pupil plane pupil plane 5 Numerical Aperture and F-number image . Classical measure for the opening: plane numerical aperture (half cone angle) object in infinity NA' nsinu' DEnP/2 u' . In particular for camera lenses with object at infinity: F-number f f F# DEnP . Numerical aperture and F-number are to system properties, they are related to a conjugate object/image location 1 . Paraxial relation F # 2n'tanu' . Special case for small angles or sine-condition corrected systems 1 F # 2NA' 6 Generalized F-Number .
    [Show full text]
  • Single-Image Vignetting Correction Using Radial Gradient Symmetry
    Single-Image Vignetting Correction Using Radial Gradient Symmetry Yuanjie Zheng1 Jingyi Yu1 Sing Bing Kang2 Stephen Lin3 Chandra Kambhamettu1 1 University of Delaware, Newark, DE, USA {zheng,yu,chandra}@eecis.udel.edu 2 Microsoft Research, Redmond, WA, USA [email protected] 3 Microsoft Research Asia, Beijing, P.R. China [email protected] Abstract lenge is to differentiate the global intensity variation of vi- gnetting from those caused by local texture and lighting. In this paper, we present a novel single-image vignetting Zheng et al.[25] treat intensity variation caused by texture method based on the symmetric distribution of the radial as “noise”; as such, they require some form of robust outlier gradient (RG). The radial gradient is the image gradient rejection in fitting the vignetting function. They also require along the radial direction with respect to the image center. segmentation and must explicitly account for local shading. We show that the RG distribution for natural images with- All these are susceptible to errors. out vignetting is generally symmetric. However, this dis- We are also interested in vignetting correction using a tribution is skewed by vignetting. We develop two variants single image. Our proposed approach is fundamentally dif- of this technique, both of which remove vignetting by min- ferent from Zheng et al.’s—we rely on the property of sym- imizing asymmetry of the RG distribution. Compared with metry of the radial gradient distribution. (By radial gradient, prior approaches to single-image vignetting correction, our we mean the gradient along the radial direction with respect method does not require segmentation and the results are to the image center.) We show that the radial gradient dis- generally better.
    [Show full text]
  • (DSLR) Cameras
    Optimizing Radiometric Fidelity to Enhance Aerial Image Change Detection Utilizing Digital Single Lens Reflex (DSLR) Cameras Andrew D. Kerr and Douglas A. Stow Abstract Our objectives are to analyze the radiometric characteris- benefits of replicating solar ephemeris (Coulter et al., 2012, tics and best practices for maximizing radiometric fidel- Ahrends et al., 2008), specific shutter and aperture settings ity of digital single lens reflex (DSLR) cameras for aerial (Ahrends et al., 2008, Lebourgeois et al., 2008), using RAW image-based change detection. Control settings, exposure files (Deanet al., 2000, Coulter et al., 2012, Ahrends et al., values, white balance, light metering, ISO, and lens aper- 2008, Lebourgeois et al., 2008), vignetting abatement (Dean ture are evaluated for several bi-temporal imagery datasets. et al., 2000), and maintaining intra-frame white balance (WB) These variables are compared for their effects on dynamic consistency (Richardson et al., 2009, Levin et al., 2005). range, intra-frame brightness variation, acuity, temporal Through this study we seek to identify and determine how consistency, and detectability of simulated cracks. Test- to compensate and account for, the photometric aspects of im- ing was conducted from a terrestrial, rather than airborne age capture and postprocessing with DSLR cameras, to achieve platform, due to the large number of images collected, and high radiometric fidelity within and between digital multi- to minimize inter-image misregistration. Results point to temporal images. The overall goal is to minimize the effects of exposure biases in the range of −0.7 or −0.3EV (i.e., slightly these factors on the radiometric consistency of multi-temporal less than the auto-exposure selected levels) being preferable images, inter-frame brightness, and capture of images with for change detection and noise minimization, by achiev- high acuity and dynamic range.
    [Show full text]
  • Vignetting Effect in Fourier Ptychographic Microscopy
    Vignetting effect in Fourier ptychographic microscopy 1,2 3,* 4 1,2 1 1,* AN PAN, CHAO ZUO, YUEGE XIE, YAN ZHANG, MING LEI, BAOLI YAO 1 State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China 2 University of Chinese Academy of Sciences, Beijing 100049, China 3 Smart Computational Imaging (SCI) Laboratory, Nanjing University of Science and Technology, Nanjing 210094, China 4 Institute for Computational Engineering and Sciences (ICES), University of Texas at Austin, Austin, TX 78712, USA *Corresponding author: [email protected] (C. Zuo), [email protected] (B. Yao) Fourier ptychographic microscopy (FPM) is a object-image relationship [13, 14]. In real experimental conditions, computational imaging technique that overcomes the however, the resolution may decrease from the center to the edge physical space-bandwidth product (SBP) limit of a of FOV [15]. Several unexpected “semi-bright and semi-dark” conventional microscope by applying angular diversity imperfect images with strong vignetting effect can be easily illuminations. In the usual model of FPM, the microscopic observed when the sample is illuminated by the LED within the system is approximated by being taken as space- “transition zone” between bright field (BF) and dark field (DF). invariant with transfer function determined by a These imperfect images, apparently, are not coincident with the complex pupil function of the objective. However, in real LSI model and could deteriorate the reconstruction quality of FPM experimental conditions, several unexpected “semi- severely. Especially the edge of FOV, there will be obvious wrinkles.
    [Show full text]
  • Vignette and Exposure Calibration and Compensation
    Vignette and Exposure Calibration and Compensation Dan B Goldman and Jiun-Hung Chen University of Washington Figure 1 On the left, a panoramic sequence of images showing vignetting artifacts. Note the change in brightness at the edge of each image. Although the effect is visually subtle, this brightness change corresponds to a 20% drop in image intensity from the center to the corner of each image. On the right, the same sequence after vignetting is removed. Abstract stereo correlation and optical flow. Yet despite its occur- rence in essentially all photographed images, and its detri- We discuss calibration and removal of “vignetting” (ra- mental effect on these algorithms and systems, relatively dial falloff) and exposure (gain) variations from sequences little attention has been paid to calibrating and correcting of images. Unique solutions for vignetting, exposure and vignette artifacts. scene radiances are possible when the response curve is As we demonstrate, vignetting can be corrected easily known. When the response curve is unknown, an exponen- using sequences of natural images without special calibra- tial ambiguity prevents us from recovering these parameters tion objects or lighting. Our approach can be used even uniquely. However, the vignetting and exposure variations for existing panoramic image sequences in which nothing is can nonetheless be removed from the images without resolv- known about the camera. Figure 1 shows a series of aligned ing this ambiguity. Applications include panoramic image images, exhibiting noticeable brightness change along the mosaics, photometry for material reconstruction, image- boundaries of the images, even though the exposure of each based rendering, and preprocessing for correlation-based image is identical.
    [Show full text]