sign in sign up
<<
Home , DXOMARK, Exposure value, Metering mode, Vignetting

Optimizing Radiometric Fidelity to Enhance Aerial Image Change Detection Utilizing Digital Single Reflex (DSLR)

Andrew D. Kerr and Douglas A. Stow

Abstract Our objectives are to analyze the radiometric characteris- benefits of replicating solar ephemeris (Coulter et al., 2012, tics and best practices for maximizing radiometric fidel- Ahrends et al., 2008), specific and settings ity of digital single lens reflex DSLR( ) cameras for aerial (Ahrends et al., 2008, Lebourgeois et al., 2008), using RAW image-based change detection. Control settings, files (Deanet al., 2000, Coulter et al., 2012, Ahrends et al., values, white balance, light metering, ISO, and lens aper- 2008, Lebourgeois et al., 2008), abatement (Dean ture are evaluated for several bi-temporal imagery datasets. et al., 2000), and maintaining intra-frame white balance (WB) These variables are compared for their effects on dynamic consistency (Richardson et al., 2009, Levin et al., 2005). range, intra-frame brightness variation, acuity, temporal Through this study we seek to identify and determine how consistency, and detectability of simulated cracks. Test- to compensate and account for, the photometric aspects of im- ing was conducted from a terrestrial, rather than airborne age capture and postprocessing with DSLR cameras, to achieve platform, due to the large number of images collected, and high radiometric fidelity within and between digital multi- to minimize inter-image misregistration. Results point to temporal images. The overall goal is to minimize the effects of exposure biases in the range of −0.7 or −0.3EV (i.e., slightly these factors on the radiometric consistency of multi-temporal less than the auto-exposure selected levels) being preferable images, inter-frame brightness, and capture of images with for change detection and noise minimization, by achiev- high acuity and . ing a balance between full dynamic range and high acu- The applications contexts for conducting this study are ity. DSLR cameras exhibit high radiometric fidelity and can detecting post-hazard damage and monitoring changes in effectively support low-cost aerial image-based change urban infrastructure. The technical context is Repeat Station detection, such as for post-hazard damage assessment.Delivered by IngentaImaging (RSI), where image capture over time occurs at nearly IP: 192.168.39.210 On: Fri, 24the Sep identical 2021 station10:11:27 points in the sky, and image registration Copyright: American Society for Photogrammetryand change and detection Remote are Sensingperformed on a frame-by-frame basis Introduction (Coulter et al., 2003; Stow et al., 2003). Due to the dynamic Consumer-oriented digital single lens reflex DSLR( ) cameras of nature of urban scenes, a challenge is to minimize noise ever increasing spatial resolution and quality are an economi- sources due to variations in illumination characteristics and cal and readily available sensor option for airborne remote scene conditions and features, sensor variability, and appar- sensing. Increasing coverage per image frame and higher im- ent image motion (AIM) motion blur, through the selection of age fidelity, along with low barriers to entry (i.e., affordability an appropriate . The goal is to automate image and ease of use) inherent to DSLR cameras, make them viable processing and analysis as much as possible, but with the ex- for many remote sensing applications. Consumer DSLR cam- pectation that a human analyst will make the final analysis of eras have features and incorporate processes that differ from whether damage or other land surface changes have occurred. traditional aerial cameras, as they are designed with photom- This study was conducted in such a way that the charac- etry in mind, rather than high radiometric fidelity associated teristics of the lens used, such as the specific or various focal with many aerial imaging sensors. length(s) and relative aperture(s), had minimal bearing on the seeks to measure light, as closely as possible replicability and adaptability to other users, can be performed to how it is perceived by the human eye, whereas radiometry without specialized equipment, and can be tailored to specific seeks to normalize or even measure the absolute spectral radi- collection parameters. The study design also should allow for ance. Having photometric accuracy as the primary concern of future models to be tested, provided that they have a camera manufactures, has led to the development of several variable aperture lens, bayer array, and similar photometric materials and processes, including specific lens to a CMOS sensor. We captured high quality images with vary- or sensor spectral coatings, and onboard image processing ing exposure parameters using consumer grade DSLR cameras, steps, to achieve greater photometric accuracy, sometimes at and the resultant image sets were utilized to empirically the expense of radiometric linearity or greater radiometric ac- address the following research questions, in the context of curacy (Lebourgeois et al., 2008). RSI-based change detection. A dearth of research articles exist that explore remote 1. With what combination of exposure settings can the sensing based on DSLR digital cameras (Clemens, 2015). These dynamic range of image brightness values be maximized, studies have tended to focus on vegetation remote sensing while achieving high image acuity? (Dean et al., 2000, Ahrends et al., 2008, Lebourgeois et al., 2008, Richardson et al., 2009), change detection for wide area Photogrammetric Engineering & Remote Sensing aerial surveillance (Coulter and Stow, 2008), and generation Vol. 84, No. 3, March 2018, pp. 149–158. of indices for soil identification (Levinet al., 2005). 0099-1112/17/149–158 In investigating these topics, past studies have outlined the © 2018 American Society for Photogrammetry Storm Hall 307B, Department of Geography, San Diego State and Remote Sensing University, San Diego, CA 92182-4493 ([email protected]). doi: 10.14358/PERS.84.3.149

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING March 2018 149

03-18 March PR Print.indd 149 2/20/2018 4:35:47 PM 2. What is the characteristic spatial trend in brightness atmospheric path radiance, and transmittance conditions. response within image frames, how do these trends vary The sites chosen for these image collections (Figures 1 and with different exposure parameters, and how well can 2) have urban scene composition, and an image capture within image trends be balanced or normalized? azimuth nearly parallel to the solar azimuth, so as to create 3. How can between image differences in radiometric bright- illumination and shadow conditions similar to those for aerial ness response due to noise be minimized and due to signal imaging. Figure 3 is the camera and set-up for station- be maximized for multi-temporal RSI pairs, by proper ary oblique imaging. Additionally, images were collected on selection of exposure parameters? a clear and a hazy day, so the dataset contains two different atmospheric conditions. We find no previous studies in the literature that evaluate the The experimental variables that pertain to the research radiometric characteristics of consumer grade DSLR cameras questions in this study are image and light me- for capturing aerial imagery in support of change detection ter measurement zones for automatically estimating the appro- applications, nor for determining best practices for optimal priate exposure level for particular ambient light magnitudes, radiometric fidelity through optimization of camera con- aperture size, and shutter speed for controlling the amount trol parameters. In addition, this study is novel in both its of light reaching the detector, ISO (a carry over acronym from ground-based image capture procedures that emulate airborne metrics of the International Standards Operation) imaging, including simulation of vertical path radiance in the for controlling the sensitivity of the detector and white bal- horizontal plane, and near replication of solar geometry, and ance for inter-band radiometric consistency. Specific variables its damage change detection metrics, which were developed pertain to each of the research questions, as not all of the vari- for measuring change detection using a type of signal to noise ables are relevant for addressing each research question. metric, and two quantitative change detection evaluation The first research question pertains to maximizing dynam- methods based on transects/profiles. ic range of a given image frame, while achieving high image acuity, for which the image exposure value (EV), measurement zones, relative aperture, shutter speed, and ISO Methods are the five relevant variables. The frame-specific EV is a major determinant of dynamic range, as it sets the camera exposure Experimental Variables and Image Capture Strategy controls to yield an image that is not over- or under-exposed, The methods chosen for this study were designed for col- thus ensuring a maximized dynamic range in the resulting lection of imagery from a terrestrial, rather than an airborne image. The light meter measurement zone configuration may platform. The rationale is that for each image collection of 86 have an impact on a maximized dynamic range as well, as unique camera exposure setting combinations, the capture the “overall” and “center weighted” settings for determin- location and view geometry must be as close to identical ing which of the detector array (and therefore, which as possible. Such a dataset would have been infeasible to parts of the scene) are used for light metering, may differ in generate vis a vis an actual aerial image acquisition withDelivered the bytheir Ingenta consistency for maximizing dynamic range. Slow shutter appropriate experimental control. IP: 192.168.39.210 On: Fri,speeds 24 Sep can 2021 create 10:11:27 image blur in actual aerial image capture, In order to simulate aerialCopyright: image collection, American various Society col -for Photogrammetrydepending on theand influence Remote Sensingof apparent image motion AIM( ). lection parameters were set to approximate illumination, AIM was calculated by multiplying the speed of the platform

Figure 1. Oblique intensity image of La Jolla Business Park scene (San Diego, California), used for the light meter testing. Specific image subset used for theRMSD testing is delineated by the black rectangle.

150 March 2018 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

03-18 March PR Print.indd 150 2/20/2018 4:35:47 PM (ms -1), by the image capture time(s), and then dividing the values for spatially corresponding pixels of multitemporal product by the GSD of the image in (meters per pixel). We image sets, with light meter measurement zones and white estimated AIM for various exposure settings that were deter- balance being the variables of relevance. The light meter mined to be optimal radiometric fidelity and image acuity, measurement zone selected may have an impact on how the based on platform altitudes and focal lengths camera assesses the scene brightness between imagery collec- used in imagery collections associated with our RSI change tions over time. On account of changes to atmospheric optical detection research (Coulter et al., 2008; Stow et al., 2016). and illumination conditions, and land cover or ground feature Images with an AIM of greater than one have the potential for changes within the scene, WB is a relevant variable to consider acuity loss resulting from ground movement that occurred when seeking to minimize temporal differences in brightness. during the shutter action. Similarly, ISO has an effect on image Having laid out the variables of interest, and how they pertain acuity, with image shot noise increasing as the camera gain is to the individual research questions, the experimental design increased, resulting in poorer image acuity. used to collect image data for testing these variables follows. The second research objective pertains to within frame Images were collected with combinations of five general trends in brightness response and how across-frame bright- exposure variables: EV, WB, light metering, ISO, and aperture ness variation can be minimized, with relative aperture and (f/stop). The collections were all performed from fixed camera ISO being the variables of relevance to this question. Lens stations and a tripod platform. The image center was fixed on aperture has an effect on vignetting within an image, with the same landmark for image collected of the same scene, to lower f-stop numbers resulting in more pronounced vignett- insure consistency in the perspective of the camera. None of ing effects. Also, variable radiance to digital number response the scenes imaged had objects within them closer than 350 m may occur across an image frame as a function of detector from the camera. The controlled constants for all images, un- sensitivity (ISO). less explicitly stated otherwise were: EV = 0, WB = automatic The third research objective involves minimizing differ- (AWB), light metering = overall, ISO =100, and (f) ences in image brightness intensity or digital number (DN) = 55 mm. Images with varying EVs were captured at ⅓ step

Delivered by Ingenta IP: 192.168.39.210 On: Fri, 24 Sep 2021 10:11:27 Copyright: American Society for Photogrammetry and Remote Sensing

Figure 2. Oblique intensity image of Mt. Soledad scene (San Diego, California), used for the light meter testing.

Figure 3. Location and position of the camera and tripod used for all of the imaging, positioned at the imaging site for the La Jolla Business Park (see Figure 1).

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING March 2018 151

03-18 March PR Print.indd 151 2/20/2018 4:35:47 PM intervals, between −3 and +3, for a total of 19 different EVs To mimic airborne imagery collection from a terrestrial captured (see Table 1). WB variable images were captured location, we attempted to approximate an equivalent path utilizing the camera presets listed in Table 1 at the EVs of 0, radiance to match a range of altitudes (750 to 1500 m) above −1 and +1, for a total of 27 images. Variable light metering ground level (AGL), that we commonly use as platform alti- tests were conducted using the “center weighted” light meter tudes for our simulated damage assessment research. Optical setting, at the EVs listed in Table 1, for a total of seven images. depth increases horizontally along with the zenith angle, from The variable ISO tests were carried out at the EVs of −1, 0, and a factor of one at zenith angle 0°, to around 40° at zenith angle +1, using the ISO values listed in Table 1, for a total of 18 im- 90° (Allen, 1973), and vertically, with a zenith angle of 0°, the ages. The variable aperture tests were conducted using 19 dif- magnitude of an object at the top of the atmosphere decreases ferent ranging from f/2 to f/16, utilizing a different by 0.28 at sea level, 0.24 at 500 m, and 0.21 at 1,000 m above camera system from all the other tests, with ISO = 50, f = 105 sea level, to 0 at around 100 km (Green, 1992). With 25 mm, four different degrees of onboard vignetting control, and percent of atmospheric scattering due to atmosphere located EVs of −⅓, ⅔, 0, and +⅔, for a total of 228 images. between sea level and 1,000 m, this would mean a horizontal Two DSLR camera systems were used in this study, a image at 1,000 m would have 25 percent less path radiance D800E full frame camera paired with a Nikon DC-Nikkor 105 than the same image captured at sea level. Given both the mm f/2 to f/16 relative aperture lens, which was used for the non-linear and temporally varying nature of optical depth, a variable aperture study, and a Sony Alpha 65 APS-C camera 25 percent reduction in path radiance was an estimate used (Bockaert, 2006) paired with a Sony f = 18-55 mm f/3.5 to 5.6 for this study. We simulated vertical imaging altitudes of 750 relative aperture lens, which was used for the variable EV, WB, m and 1,500 m AGL, such that the 565 m to 1,125 m in the light meter, and ISO studies. The Sony a65, shown setup for horizontal plane, above ground level. our experimental imaging in Figure 3, would have been the In addition to images datasets generated for the change only camera utilized for this study, however the widest aper- detection noise minimization tests (described above), an- ture usable to capture the 55 mm images, f/5.6, is an aperture other set was created that simulated infrastructure damage. at which vignetting effects are negligible. For this reason, Simulated cracks were used to test the sensitivity of different the Nikon D800E with a wider f/2 aperture lens with more capture settings in detecting cracks in a wall or road as prox- pronounced vignetting effects was utilized for the aperture ies for infrastructure damage. Image pairs taken of simulated testing (DXOMARK, 2012). cracks simulated were analyzed to determine the settings for which the smallest crack can be resolved from difference Table 1. Experimental camera exposure variables and their images and to examine the influence of EV settings on crack settings. detection. To assess the ability to detect damage signals, simu- EVs White Light Metering Relative lated cracks made out of black painter’s tape were placed on (19) Balance (9) EVs (7) ISO (6) aperture (19) a concrete wall with widths approximately equal to one, two, three, and four pixels (GSD = 4.5 cm). Vertical cracks were cre- -3 AWB -3 100 f/2 Delivered byated Ingenta with varying widths, equal to one, two, three, and four -2 ⅔ 2500K -2 IP: 192.168.39.210200 f/2.2 On: Fri,times 24 Septhe estimated 2021 10:11:27 image ground sampling distance (GSD), -2 ⅓ 3500K Copyright:-1 American400 Societyf/2.5 for Photogrammetryand diagonal cracks and Remote were created Sensing equal to one, two, and four -2 4500K 0 800 f/2.8 times the estimated GSD, using matte black tape. An oblique -1 5500K +1 1600 f/3.2 imagery collection was performed with the Sony camera mounted on a tripod from the top of building 42 m vertically, 0 6500K +2 3200 f/3.5 960 m horizontally distant from the concrete wall. A second +1 7500K +3 f/9 collection (repetition of EV settings) made after the tape had +2 8500K f/10 been removed. Image pairs for each EV setting were regis- +2 ⅓ 9500K f/11 tered and then subset to the extent covering the simulated cracks and their surrounding background (concrete wall). The +2 ⅔ f/13 images for this test were collected with only a single WB of +3 f/14 AWB, compared to the nine WB settings captured for the noise f/16 minimization test imagery, all other settings utilizing the con- trolled constants from the noise minimization image set. The The Nikon D800E was used to test across-image variability ideal EV level is that which enables detection of the smallest during a single collection, utilizing a reflectance calibration simulated cracks, with the greatest temporal brightness differ- panel as a surface of uniform brightness. The reflectance ence across the simulated crack features. panel was positioned perpendicular to the sun, with the camera positioned as close to perpendicular with the sun and reflectance panel as possible, without having the shadow of Data Analysis the camera in the resulting image. Images were captured in Automated change detection products are typically based on the mid-afternoon, to lessen the chances of accidental overex- image differencing of one or more wavebands or transform im- posure when capturing images with a high f-stop. ages for a RSI pair (in our case), creating an output a raster with The Sony camera captured four image sets for two different band values representing the magnitude and sign of DN value scenes in support of dynamic range and spatial acuity tests changes. The most appropriate (if not optimal) imagery collec- with variable EV, WB, light metering, and ISO. Two collections tion settings were those combination of settings which produce were carried out in the afternoon with a solar elevation of images with the lowest root mean square difference (RMSD) val- 45°, and two collections were carried out in the late afternoon ues for DNs in the difference images, for test images or subsets with a solar elevation of 30°, to achieve variations in illumi- where scenes contained few actual land surface changes. nation conditions and degrees of shadowing. Images were Prior to image analyses, test images were downloaded, captured on clear and hazy days, and yielding image data sets sorted, and assessed for quality, for both the noise minimiza- influenced by two differing atmospheric optical conditions. tion and simulated crack signal maximization assessments. The sites chosen were selected to capture differing urban Test image pairs used for dynamic range and light metering, scene compositions, and to achieve an image viewing azimuth vignetting, and simulated crack signal assessments, required angle parallel to the solar azimuth at time of collection.

152 March 2018 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

03-18 March PR Print.indd 152 2/20/2018 4:35:47 PM registration (or co-alignment), which was performed vis a vis weighted metering, compared with the seven images captured the “image translation, scale, and rotation”, alignment tool using overall weighted metering. The two zonal calculation in ® CC. Difference images were generated methods are referred to by the camera’s manufacturer (Sony), as from these registered pairs by subtracting the per band DN and “multi-segment” and “center weighted”, both methods utilize intensity (of the hue-saturation-intensity transform) values of different proprietary formulas, and as such, the computation the pre-damage image from those of the post-damage image. differences between them is unknown. Whichever metering method had APEX values that more consistently aligned with Dynamic Range Assessment the EV values was deemed the more consistent light metering Dynamic range was assessed for each waveband, for all 19 EV method, and the preferable camera settings choice. levels captured per imagery collection, and for two different light metering zonal calculation methods. The optimal EV Simulated Damage Assessment and light metering method, were assessed as the level which Crack detection was assessed by analyzing pixel transects/ produced the largest DN value range, the lowest RMSD value, profiles across the simulated crack features and quantitatively and the lowest pixel count of truncated 0 or 255 value DNs by generating a type of signal-to-noise ratio (SNR) metric. The for each band. The “lowest” and “highest” DN value bins had signal was quantified as the spatial gradient (forward difference to contain at least 10 pixels to avoid counting random pixel operator) of bi-temporal image difference values calculated for noise that can artificially expand the dynamic range. The the pixel transects crossing the crack features. Noise was quan- dynamic range of the different EV 0 images was compared to tified as the RMSD of bi-temporal image difference values for a determine if the center, or overall weighted light metering subset of the background surface surrounding but not including method consistently produces images with greater dynamic the cracks. The SNR value is the quotient of the signal divided range. This test was performed with both the 8-bit and 12-bit by the noise metric for bi-temporal subset pairs for each EV trial. RAW images, to determine if the optimal EV level for 8-bit im- ages, is the same EV level for 12-bit images. Intra Frame Brightness Variation Assessment Results The effects of white balance were assessed with the AIM of de- Dynamic Range Assessment termining the optimal camera setting or processing method, to Dynamic range was assessed for 133 pairs of images in 8-bit achieve temporal consistency between bi-temporal image pairs. and 12-bit radiometric resolution, all captured on the same Temporal inconsistency across dates, as a result of uninten- time of day, spanning two months and three locations, and tional variations in WB , could result in a bias for 19EV settings in ⅓ increments between -3EV and +3EV. or offset of DN values in image differencing products. The dif- Dynamic range values for the 12-bit RAW images were re- ference between the AWB color temperature for each collection, corded as 16-bit containers, meaning that the values should was used to establish the typical degree of variation, between have filled histogram bins contiguously, taking up 4,096 of the WB on different dates. The minimum and maximum color the 65,536 total bins. However, the RAW file format used by temperature values from all images collected, to determineDelivered if by Ingentathe camera (Sony ARW format) automatically applied lossy fixing the color temperature offers IP:substantial 192.168.39.210 benefit. On: Fri, 24compression Sep 2021 10:11:27to all of the images, which had the effect of non- The influence of relativeCopyright: aperture American settings Societyon vignett for- Photogrammetrycontiguously and spreading Remote DN Sensing values from the 12-bit source data, ing was assessed. This involved stacking a large number of across the full 16-bit container value range, leaving empty DN images of a uniform reflectance target captured for multiple bins between ones containing data. relative aperture settings, and extracting diagonal pixel DN All 133 images contain pixels with the maximum (max) profiles to estimate radial brightness trends that represent the value (65535), while 49 images also have pixels with the min vignetting effect. The brightness trend images can be used value (0). Images that have pixels in both the max and min to create an anti-vignetting correction mask, as an alterna- bins, do not necessarily represent the optimal dynamic range, tive to the correction masks provided by the camera and lens as many of those images were highly underexposed and had manufacturers (Stow et al., 1996). Radial trends were com- the majority of pixels spread across only a handful of bins. pared between pixel profiles generated with the four different For this reason, the standard deviation of both the 8-bit and internal camera vignetting compensation settings (none, low, 12-bit images were calculated in order to contextualize the medium, and high). dynamic range differences. A plot of DN values for 12-bit images, shown Figure 4a, illus- Image Visual Acuity Assessment trates that the highest standard deviation of intensity values for The influence of ISO setting on image acuity was visually the entire image, occurs around EV 0, with the images between evaluated, in order to determine at what value threshold im- EVs −3 and −2/3, having maximum ranges. Visually, images age acuity begins to decline. This was accomplished by taking between EV −1 to EV 0 exhibit the highest degree of acuity, with images with different ISO values, randomizing them, and then this range of values achieving a good balance between maxi- trying to determine which images are visually noisier than the mized dynamic range and a wide distribution of DN values. image captured with ISO 100. The 8-bit images exhibit a trend similar to the 16-bit image Temporal Consistency tests, and are visually indistinguishable from their 12-bit coun- Magnitudes of noise in bi-temporal pairs were assessed for terparts. A 2nd degree polynomial trend line was used, because 2 each band, for all 19 EV levels captured, per imagery collec- the trend was parabolic, with the r included to highlight the tion, and for two different light metering zonal calculation difference between the 8-bit and 12-bit trends. As seen in Fig- methods. For both the comparison of the 19 EV levels and ure 4b, EV −1 to EV 0, as with the 12-bit images, seem to be the light meter weighting methods, red, green, blue, and intensity ideal range for achieving both a maximized dynamic range, a RMSD values were derived. The image settings which yielded wide distribution of DN values, and the highest degree of acuity. the lowest calculated RMSD for each band represented the Intra-frame Brightness Variation Assessment - White Balance Influence optimal camera settings. Across 20 collections on 10 dates, at three sites, over three For seven of the 19 EVs evaluated, light metering perfor- months, the color temperature range was 5150K to 5300K mance was assessed by comparing how consistently the APEX utilizing the onboard camera AWB. This variation of 150K value recorded in the image metadata as “Brightness Value” will cause minor hue, saturation, and intensity fluctuations and EV offsets, maintain parity with each other for the center between dates, if a set value is not fixed onboard the camera,

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING March 2018 153

03-18 March PR Print.indd 153 2/20/2018 4:35:47 PM (a)

(b)

Delivered by Ingenta IP: 192.168.39.210 On: Fri, 24 Sep 2021 10:11:27 Figure 4. (a) Dynamic rangeCopyright: portrayed American by minimum Society and standard for Photogrammetry deviation of intensity and Remote digital Sensing number values for the 16- bit container as a function of EV offset settings; and (b) Dynamic range portrayed by minimum and standard deviation of intensity digital number values for the 8-bit container as a function of EV offset settings.

Figure 5. f/2 to 16 (19 different apertures), captured with Figure 6. EVs 0 and −⅓ imaged using f2 with no, low, normal no vignetting reduction applied at EV 0; plotted by DN of and high vignetting correction, plotted by DN of intensity and intensity and distance from the image corner. distance from the image corner.

or if RAW images are not post-processed using the same value. balance between selecting an f-stop that allows for proper im- This variation of 150K is relatively small, compared with the age exposure when paired with a quick shutter speed, while fixed white balance values on the camera that ranges between maintaining a tolerable level of image vignetting, will depend 2000K and 10000K. on to what degree brightness variation in the corners of the frame may be detrimental in the intended application. When Intra-frame Brightness Variation Assessment - Aperture Influence implementing RSI with same DSLR camera, frame centers are Intra-frame brightness variation stemming from vignetting nearly identical for both images of the bi-temporal pair, such and other off-axis optical effects, was more pronounced for that any vignetting effects will be manifested similarly and images with a wider aperture (lower f-stop number), as theory impacts of brightness follow-off on change detection should would suggest. As seen in Figure 5, irrespective of the f-stop be minimal. For almost all consumer digital cameras, vignett- used, or the degree of onboard vignetting compensation used, ing mitigation is usually a component of the all images exhibit some degree of vignetting in the approxi- development process. mately on the image periphery. Striking the appropriate

154 March 2018 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

03-18 March PR Print.indd 154 2/20/2018 4:35:47 PM The effects of mitigating vignetting on dynamic range are Temporal Consistency - Background Noise Reduction difficult to document, as the brightness fall off artificially In order to evaluate the EV bias that provided the greatest tem- enlarged the range on the low end, with no effect on the high poral consistency between dates, images captured at two times end for the images tested. As shown in Figure 6, slightly of day, on two different dates, of the same scene are compared underexposed images had less overall vignetting (by 1 to 2 against each other. Over the course of an approximately 20 percent across all f-stops tested). The vignetting correction min imagery collection, conditions in the scene were mostly process built into the camera, as shown in Figure 6, decreased static, and no damage or change features were simulated. vignetting slightly, but mitigated no more than approximately The assumption is that most pixels detected as changed are 10 percent of the DN value drop between 0.67 and 0.77. representative of variations (i.e., noise) caused by differential camera settings, rather than actual changes in the scene. Image Visual Acuity Assessment - ISO Bi-temporal image pairs of the same exposure bias, from The effects of ISO settings on visual acuity were assessed by the same time of day, were used to create difference images, repeating the crack detection pixel profile experiment using from which their RMSD was plotted in Figure 7. The lowest image pairs with higher ISO values (200 to 6400). a visual as- RMSD value occurring at EV −3 indicates the lowest image sessment of various ISO values, across several images, yielded noise, whereas the peak (“noisiest”) RMSD value is around the set of observations in Table 2. These observations are EV +0.7. The cause of the two outlier RMSD values is indeter- consistent across image sets captured at different times of day, minate, although, it is suspected that they stem from lower times of year, and of different scenes. Image (“shot”) noise visual acuity resulting from movement of the camera. occurring at higher gain settings is sufficiently minor and ran- dom enough relative to random fluctuations created by image Temporal Consistency - Light Metering Method alignment and light meter fluctuations, only minor influences Light metering consistency was compared between center on detectability of the simulated crack test images is observ- weighted and overall weighted metering methods, across a able between image pairs with different ISO values. total of 755 images, captured across four dates in June, at two Using a common ground speed of 120 km/h (75 mph) sites, and at two times of day. For every collection, EV 14.7 (3,333 cm/sec) with an anticipated GSD of 7.5 cm, and an ideal was the most common EV for images with an exposure bias of AIM of less than one, we determined that that shutter speeds ± 0, as shown in Table 3. Thus, it was selected as the EV level faster than 1/407 sec (rounded to 1/400 sec) are needed to from which to calculate the offset between EV bias and camera avoid potential motion blur. Shutter speeds faster than 1/400 configuration EV, relative to 0. were selected by the camera auto-exposure program during As shown in Table 4, center weighted metering outper- the imagery collections, although a fixed shutter speed mini- forms overall weighted metering by 19 percent for images mum value (preferable) and/or aperture priority should be set with no offset at all, and by 2 percent for images with an on the camera to insure a consistently adequate shutter speed. offset of ± 0.3EV. The selected value of 14.7EV matched well with both the 30° and 45° solar elevation angles, with Table 2. Qualitative evaluation of the image noise present in 93 percent and 96 percent falling within ± 0.3EV of 14.7EV, Delivered by Ingentarespectively. images of differing ISO levels. IP: 192.168.39.210 On: Fri, 24 Sep 2021 10:11:27 ISO Qualitative ObservationsCopyright: American Society for PhotogrammetrySimulated Damage and Remote Assessment Sensing - Crack Detection Examples of simulated pre-damage and post-damage image 100 High visual acuity, no visible image artifacts, sharp object edges subsets are shown in Figures 8a and 8b, respectively. Image 200 High visual acuity, no visible image artifacts, sharp object pairs were used to create difference images, from which a edges, indistinguishable from ISO 100 trio of 74 horizontal pixel profiles were extracted, and then 400 Good visual acuity, noticeable image noise in areas of high averaged into a single row of 74 pixels (to minimize random homogeneity noise effects). An analysis of spatial acuity of the images was done by examining the pixel profiles, while anSNR metric was 800 Average visual acuity, image artifacts present throughout based on calculating the difference magnitude and spatial gra- image, borderline usability for detection of small objects dient across the simulated cracks, relative to the background >1600 Visual acuity loss is significant, large image artifacts noise (RMSD of difference image for the non-crack area). throughout the image, likely unusable for applications Figure 9a shows the difference magnitude of the pixel requiring detection of small objects profiles, with the value of each pixel in the profile squared,

Figure 7. Root means square difference (RMSD) intensity values derived from bi-temporal image pairs from 30° and 45° solar elevation June 25 and 26 collections at La Jolla site (Lower values mean lower noise levels).

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING March 2018 155

03-18 March PR Print.indd 155 2/20/2018 4:35:47 PM Table 3. EV bias +/- 0 images from four Table 4. The APEX offsets by , relative to the target EV value of 14.7. collection dates, at two times per day, APEX EV Offset Metering Mode Count Percent of mode of two different scenes, using two light metering modes. -0.6 Center-weighted average 1 2% -0.3 Center-weighted average 2 4% APEX EV Count Percentage of Category 0 Center-weighted average 40 71% 13.7 1 1% +0.3 Center-weighted average 12 21% 14.3 32 17% +1.3 Center-weighted average 1 2% 14.7 110 59% -1 Multi-segment 1 0% 15 40 21% -0.6 Multi-segment 11 2% 15.3 3 2% -0.3 Multi-segment 125 20% 15.7 1 1% 0 Multi-segment 329 52% All 187 +0.3 Multi-segment 139 22% +0.6 Multi-segment 18 3% +1 Multi-segment 5 1% summed, and then divided by 74. Figure 9b shows the spatial gradient (forward differ- +1.3 Multi-segment 1 0% ence) of the pixel profiles, with the value of Total Center-weighted average 56 a pixel subtracted from the adjacent pixel, Total Multi-segment 629 squared, summed, and then divided by 74. For all three of these metrics, EV -1 and EV Discussion and Conclusion -1.3 appear to have the strongest signal, where higher values This goal of this study is to determine how to optimize are ideal, as they indicate a greater degree of detection of radiometric parameters of DSLR cameras to enable reliable detected change across the cracks. The anomalous drop in the image-based change detection, through the exploration value of EV 0 is likely due to lower image acuity for one of the of three research questions: (1) with what combination of images in the image pair, while the cause of the acuity loss is exposure settings can the dynamic range of image brightness uncertain. Even so, a clear trend in the relationship between EV values be maximized, while achieving high image acuity?; (2) setting and crack detection is observable in Figures 9a and 9b. what is the characteristic spatial trend in brightness response As shown in Figure 10, the signal and noise RMSD values within image frames, how do these trends vary with differ- tend to co-vary with the EV bias values. The signal RMSD ent exposure parameters, and how well can within image however, exhibits a trend with EVs −1.7 to −1 exhibiting the trends be balanced or normalized?; and (3) how can between highest signal, and some of the higher SNR scores, suggest- image differences in radiometric brightness response due ing that this narrow range of EV biases is preferable forDelivered crack byto Ingentanoise be minimized and due to signal be maximized for detection. Visually, within the change IP:detection 192.168.39.210 images and On: Fri,multi-temporal 24 Sep 2021 RSI 10:11:27 pairs, by proper selection of exposure pixel profiles, cracks the equivalentCopyright: width American of a single Society pixel for of Photogrammetryparameters? These and questionsRemote Sensing were tested and analyzed within GSD are readily identifiable as change objects in most of the an experimental framework that analyzed the influence of five image pairs (Figure 8d). variables, EV, WB, light metering, ISO, and relative aperture, on

Figure 8: (a) Subset of oblique intensity image used for simulated crack detection (a) experiment “before” (time −1) image of a concrete wall (with black rectangular mask over a foreground object) difference; (b) Subset of oblique intensity image used for simulated crack detection experiment “after” (b) (time−2) image containing black tape segments representing cracks of varying widths; ( c) Subset of oblique intensity image used for simulated crack detection experiment “after” image subset for the bounding (c) rectangle encompassing simulated cracks (“signal subset”) and other area (“noise subset”) masked in black; and (d) Subset of oblique intensity image used for simulated crack detection experiment difference (d) image of the “signal subset.”

156 March 2018 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

03-18 March PR Print.indd 156 2/20/2018 4:35:48 PM image radiometry, temporal consistency, and simulated crack Test results for light metering methods indicate that damage detection. “center weighted” outperforms “overall” light metering. This When examining EV bias and its effects on dynamic range, seems intuitive given the smaller area used for illumination we found that the minimum intensity and standard deviation calculation by center weighted method than by the overall bin values for the 8-bit and 12-bit images are nearly identical. method, and lower potential for the periphery of the scene to Identical results were not expected, given that both the 8-bit cause calculation variability. However, this leaves the sensor and 12-bit images underwent different, lossy compressions, operator with a choice between more consistent light meter- performed onboard the camera, which compressed the dy- ing, and light metering which best takes into account varia- namic range for the 8-bit images, and created gaps in the dy- tions in illumination along the periphery of the scene. namic range of the 12-bit images, as explained in the Results Detector gain as measured by ISO was the camera control Section. This prevented direct comparability between 8-bit variable that proved the most difficult to compare quantita- and 12-bit dynamic ranges, as the linear 1:16 ratio between tively, and no clear trend emerged when comparing images the two was disrupted. Images close to, or with a positive with simulated cracks across varying ISO levels. Visually, bias above neutral exposure, had the highest standard devia- images become grainier, with increased shot noise beyond ap- tion of image intensity values, indicating that they had the proximately the ISO 200 level, with marginal image degrada- widest distribution of DN values. These images did not have tion up to about ISO 800. as high of an overall dynamic range however, as the images The experimental tests to measure the effects of relative with a exposure bias of -1 or below did. The balance aperture value on vignetting yielded variable results, de- between a full dynamic range, a wide distribution of values, pending on the intended image use. If images are captured and high visual contrast seems to occur at either the −0.7 or with an area of interest in the center 40 percent of the image, −0.3EV exposure bias. rather than for generating an image mosaic where most of As indicated in the literature review, white balance is a an image frame is used in the final product, the use of wider potential source of unintentional image variance if not con- apertures with more pronounced vignetting effects, is accept- trolled as a variable. Within this study, the variation in white able. This is because the areas of brightness fall off primarily balance was relatively small between all of the test images. occur within the outer portions of an image frame. In this This leads to minor variations in DN values if not controlled case, the recommended f/stop should be a higher to minimize as a postprocessing variable. brightness fall-off effects. Within the frame-based change

Delivered by Ingenta IP: 192.168.39.210 On: Fri, 24 Sep 2021 10:11:27 Copyright: American Society for Photogrammetry and Remote Sensing

(a) (b) Figure 9 (a). Effects of EV bias setting on simulated crack detection as measured by the squared DN change difference magnitude, and (b) Effects of EV bias setting on simulated crack detection as measured by the spatial gradient of the difference between adjacent pixels, squared (right), of the simulated crack pixel profiles byEV bias.

Figure 10. RMSD intensity values of bi-temporal image pairs extracted for the ‘signal’ (image difference of the crack area) and ‘noise’ (image difference of concrete wall background) subsets of Figure 8c, for the simulated crack detection assessment.

PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING March 2018 157

03-18 March PR Print.indd 157 2/20/2018 4:35:48 PM detection context of this study, the periphery of the image is References not as important, and brightness fall-off should be consistent Ahrends, H.E., R. Brügger, R. Stöckli, J. Schenk, P. Michna, F. between bi-temporal pairs. Automatic mitigation of camera Jeanneret, H. Wanner, and W. Eugster, 2008. Quantitative vignetting did not produce large or noticeable image artifacts phenological observations of a mixed beech forest in northern in the process, and had an effectiveness proportional to the Switzerland with , Journal of Geophysical Research, 113:1–11. initial degree of vignetting. When capturing airborne imagery, selection of an aperture, or aperture range, should be carefully Allen, C.W., 1973. Astrophysical Quantities, England, 125 p. considered relative to the sensor gain and shutter speed, and Bockaert, V., 2006. Sensor Sizes, URL: https://web.archive.org/ the collective effect of all three on image acuity and intra- web/20110415004138/http://www.dpreview.com/learn/?/Glossar y/Camera_System/sensor_sizes_01.htm (last date accessed: 05 frame brightness consistency for the ease of RSI frame to frame January 2018). registration and difference image creation. Clemens, S.R., 2015. Procedures for Correcting Temporal consistency in image brightness was evaluated Imagery Acquired by the AGGIEAIR Remote Sensing Platform, relative to how much background image noise was present in Utah State University. images with different capture parameters. The trends in Figure Coulter, L., D. Stow, and S. Baer, 2003. A frame center matching 7 look very similar to the intensity standard deviation results technique for precise registration of multitemporal airborne in Figures 4a and 4b, except that lower values were preferable frame imagery, IEEE Transactions on Geoscience and Remote in the former, and higher values preferable in the latter two. Sensing, 41(11):2436–2444. EV bias values furthest from +0.7 EV are the best choice for low Coulter, L., and D. Stow, 2008. Assessment of the spatial co- noise change detection images, excluding the images with an registration of multitemporal imagery from digital exposure bias of >+0.7, which had low image noise stemming cameras in the context of detailed change detection, Sensors, from high numbers of truncated (255 value) pixels. Images 8(4):2161–2173. with negative exposure biases greater than −2 did not exhibit Coulter, L., D. Stow, Y.H. Tsai, C. Chavis, C. Lippitt, G. Fraley, and truncated values, but had a small standard deviation of pixel R. McCreight, March 2012. Automated detection of people and values and low contrast, which lowered the measured differ- vehicles in natural environments using high temporal resolution ence in image noise, as shown in Figures 4a and 4b. airborne remote sensing, Proceedings of the ASPRS Annual Detection results from images with simulated cracks Conference, March 2012, pp. 1–13. exhibit the same optimal EV bias as several of the previous Dean, C., T.A. Warner, and J.B. McGraw, 2000. Suitability of the tests, with the values nearest to −1.3 and −1EV, yielding the DCS460c colour digital camera for quantitative remote sensing analysis of vegetation, ISPRS Journal of Photogrammetry and best detection results. Small linear features such as cracks in Remote Sensing, 55(2):105–118. an asphalt road within the scene (which were present in the DXOMARK, 2012. Sigma 105mm F2.8 EX DG Macro Nikon mounted before and after images) were also readily detectable with the on Nikon D800E: Tests and Reviews, URL: https://www. −1.3 and −1EV images. .com/Lenses/Sigma/Sigma-105mm-F28-EX-DG-Macro- The experiments designed and executed in this thesis Nikon-mounted-on-Nikon-D800E__814 (last date accessed: 05 study were conducted in such a way, that the methodsDelivered should by IngentaJanuary, 2018). be useful for many remote sensing professionalsIP: 192.168.39.210 looking to On: Fri,Green, 24 SepD.W.B., 2021 1992. 10:11:27 Correcting for atmospheric extinction, use consumer DSLR cameras.Copyright: However differentAmerican scene Society char -for PhotogrammetryInternational and Comet Remote Quarterly Sensing, 14:55–59. acteristics, analysis objectives, and sensors, may limit the Lebourgeois, V., A. Begue, S. Labbe, B. Mallavan, L. Prevot, and applicability of the results discussed here. B. Roux, 2008. Can commercial digital cameras be used as Additional work utilizing a similar testing framework, but multispectral sensors?, A crop monitoring test, Sensors, a different camera system, and scene, would be helpful in de- 8(11):7300–7322. termining which of the results obtained in this study pertain Levin, N., E. Ben-Dor, and A. Singer, 2005. A digital camera as a tool to specific parameters of the study, and which are more wide- to measure colour indices and related properties of sandy soils ly applicable to general end users. Specifically, with regard to in semi-arid environments, International Journal of Remote the simulated damage crack detection, the testing of datasets Sensing, 26(24):5475–5492. that include actual damage, and other types of damage such Richardson, A.D., B.H. Braswell, D.Y. Hollinger, J.P. Jenkins, and as rubble and object deformation, may prove insightful. The S.V. Ollinger, 2009. Near-surface remote sensing of spatial capture of images on overcast days, at high or lower latitudes, and temporal variation in canopy phenology, Ecological Applications, 19(6):1417–1428. and from airborne platforms could also be used to address ad- ditional questions about the effects of those factors on image Stow, D., A. Hope, A. Nguyen, S. Phinn, and C. Benkelman, 1996. Monitoring detailed land surface changes from an airborne capture best practices. multispectral digital camera system, IEEE Transactions on This study is novel in its focus on the radiometric char- Geoscience and Remote Sensing, 34:1191–1202. acteristics of DSLR cameras, its ground-based image capture Stow, D., L. Coulter, and S. Baer, 2003. A frame centre matching testing approach, and its analytical damage change detection approach to registration for change detection with fine spatial context, DSLR cameras offer a convenient and low-cost solu- resolution multi-temporal imagery, International Journal of tion to many remote sensing applications for which a large Remote Sensing, 24(19):3873–3879. format imaging radiometers are beyond users’ operational, Stow, D.A., L.L. Coulter, G.R. MacDonald, and C.D. Lippitt, 2016. budgetary, and/or technical capabilities. We find that expo- Evaluation of geometric capture and processing elements in sure biases slightly less than the auto-exposure selected EV the context of a repeat station imaging approach to registration bias ±0, are preferable for change detection. Additionally, an and change detection, Photogrammetric Engineering &Remote ISO less than 200 is preferable, with an ISO of 100 or less being Sensing, 82:775–788. ideal. As for the light metering, shutter speed, and aperture, we did not find an optimal combination of camera settings which performs best in all scenarios. Instead, we found differ- ent settings are preferable, depending on the estimated AIM of the collection, the illumination of the scene, and the spectral characteristics of the objects of interest in the scene. DSLR cameras exhibit high radiometric fidelity and can effectively support low-cost aerial image-based change detection, such as for post-hazard damage assessment.

158 March 2018 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING

03-18 March PR Print.indd 158 2/20/2018 4:35:48 PM