Characterizing a Debris Field Using Digital Mosaicking and CAD Model Superimposition from Underwater Video
Total Page:16
File Type:pdf, Size:1020Kb
Characterizing a Debris Field Using Digital Mosaicking and CAD Model Superimposition from Underwater Video Jay M. Vincelli, Fatih Calakli, Michael A. Stone, Graham E. Forrester, Timothy Mellon, and John D. Jarrell Abstract Identifying submerged objects is critical for several disciplines For instance, the camera on an ROV can only capture a limited such as marine archaeology and search and rescue. However, view of the seabed. Even with wide-angle lenses, the field of identifying objects in underwater searches presents many chal- view in focus is only a small portion of the total surround- lenges, particularly if the only data available to analyze is poor- ings. When the seabed contains many objects of interest over quality video where the videographer did not plan for photogram- a widely-spaced area, this can be problematic. Without an metric techniques to be utilized. In this paper, we discuss the use expanded view of the proposed debris field, objects and their of adaptive sampling of the underwater video to extract sharp ratiometric relationships to each other cannot be analyzed. still images for stitching and analysis, and creating mosaicked While it has been possible to extract still frames, or images, images by identifying and matching local scale-invariant feature from video for a long time, still frames can suffer from the ef- transform features using computationally efficient algorithms. fects of compression due to interlacing and distortion (Negah- Computer aided design models of suspected aircraft components daripour and Khamene, 2000). Without intensive processing, were superimposed, and a feature common in multiple mo- a comparison between single images can show qualitative saicked images was used to identify a common feature between relationships between objects but often lacks meaningful purported objects to assess goodness of fit. The superimposition analytical observation. Furthermore, each image is taken at a method was replicated using landing gear from a reference air- different point of view, skewing potential analysis. In some craft and a rope of known dimensions, and favorably compared cases, imagery from different perspectives is desired, such as against the remotely operated vehicle (ROV) analysis results. in stereo-photogrammetry, where common points are identi- fied on each image, and rays can be constructed to each point Delivered by Ingentato triangulate the position of the points and allow for three-di- IntroductionDocumenting seabed environments,IP: 192.168.39.151 mapping On: the Thu, mensional30 Sep 2021 reconstruction. 05:15:31 Using photogrammetry tools such seabed, and identifying submergedCopyright: objectsAmerican is critical Society for for sever Photogrammetry- as Agisoft Photoscan and Remote (Agisoft, Sensing LLC, St. Petersburg, Russia) al disciplines including marine archaeology, geology, biology, allows for orthographic reconstruction in some situations. search and rescue, offshore drilling, and shipping industries. Here, we present an alternate approach for instances when Establishing positive identification of objects in underwater the method of video collection, and conditions of the water, searches presents many challenges. The costs involved in mean that photogrammetry cannot successfully be employed. search and recovery operations make false-positive identifica- We developed a method that allows us to retrospectively tions a pricey and protracted error. Traditionally, a dive team identify objects and scale them in situations where no scaling or a remotely operated underwater vehicle (ROV) will search device was used during the video recording and where there the proposed area for pieces of potential wreckage in a pre- is erratic or unplanned tracking of the camera. This is typical determined search pattern (Lirman et al., 2007; Zhukovsky et when the video recording was not intended to be used for al., 2013). Underwater mapping is still largely carried out by object analysis. We illustrate this approach using a case study, manual surveys and performing distance-based measurements in which underwater ROV footage with an irregular search (Telem and Filin, 2013; Ruppé and Barstad, 2002). These pattern and no method for scaling images was used (Figure tasks become much more difficult when physical contact with 1). The approach uses image processing techniques to extract the studied objects is not possible, and when scaling markers frames from HD video, de-interlace, remove time stamps, and are absent. In these circumstances, an HD video recording will stitch the remastered still frames together to create a mo- often be used while executing search patterns (Negahdaripour saicked debris field with a single perspective. We further pre- and Khamene, 2000). On its own, high-definition video can sented a strategy to use an object of known scale to measure be a powerful tool for identification but lacks many features other proposed pieces of wreckage and aid in identification. useful for characterizing a debris field (Camposet al., 2014). Background In this case study, we describe using the combination of im- Jay M. Vincelli is with Materials Science Associates, 315 age processing and mosaicking techniques using underwater Commerce Park Rd., Unit 1, North Kingstown, RI 02893 video as source data to assess the geometry of objects purport- ([email protected]). ed to be from the 02 July 1937 crash site of Amelia Earhart’s Fatih Calakli, Michael A. Stone, and John D. Jarrell are with lost airplane, a Lockheed Electra Model 10E, construction Materials Science Associates, 315 Commerce Park Rd., Unit 1, number 1055, off of the island of Nikumaroro in the western North Kingstown, RI 02893; and Brown University, 182 Hope St., Providence, RI 02912. Photogrammetric Engineering & Remote Sensing Vol. 82, No. 3, March 2016, pp. 223–232. Graham E. Forrester is with the University of Rhode Island, 0099-1112/16/223–232 Center for Biotechnology and Life Sciences, 120 Flagg Road, © 2016 American Society for Photogrammetry Kingston, RI 02881. and Remote Sensing Timothy Mellon is an independent consultant. doi: 10.14358/PERS.82.3.223 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING March 2016 223 03-16 March Peer Reviewed Revised.indd 223 2/17/2016 12:12:40 PM Delivered by Ingenta IP: 192.168.39.151 On: Thu, 30 Sep 2021 05:15:31 Copyright: American Society for Photogrammetry and Remote Sensing Figure 1. The tubular structure illustrates the camera trajectory derived from underwater ROV video, which consists of tightly-spaced, multi-colored rectangles, where each rectangle represents the camera pose in different frames of the video. The sparse point cloud (granular data) represents the view of the seabed generated from the underwater ROV video. Figure 2. Amelia Earhart’s Lockheed Electra Model 10E aircraft. Scanned from Lockheed Aircraft Since 1913 by René Francillon (Photo credit: USAF). Pacific Ocean. This airplane has an overall length of approxi- crash site by a third party, at a depth of 150 to 300 m. We mately 11.8 m, a wingspan of 16.8 m, and a height of 3.1 m received the video for analysis retrospectively, and we were (Figure 2). Two of the objects proposed to be seen in the video tasked with extracting as much information as possible from are the front landing gear and the rear wheel. Historical docu- the video footage itself. During an internal review of the ments were reviewed, and the rear tire was identified in a 19 video, two objects resembling the front and rear landing gear May 1937 aircraft inspection report to be a “Goodyear 16×7” were identified. The high levels of sedimentation and/or tire. Exemplar front and rear landing gears from an extant calcareous growth covering the purported objects prevented Lockheed Electra Model 10E, construction number 1042, were positive identification of the objects. However, meaningful provided for dimensional and visual analysis. analysis could still be performed to identify whether the un- Video was recorded during a ROV search of the suspected derwater objects seen in the video are consistent with objects 224 March 2016 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING 03-16 March Peer Reviewed Revised.indd 224 2/17/2016 12:12:43 PM from the airplane. Due to the remoteness of the crash site and (Gracias et al., 2003; Gracias and Santos-Victor, 2001). difficulty involved in safely retrieving the objects, the objec- Image mosaicking technology has many applications. Appli- tive of this study was to assess the geometry of the purported cations utilizing panoramic techniques are widely available for airplane components to determine whether additional investi- smartphones, where overlapping photographs are taken from gation of these objects, such as retrieval, is merited. a single position in three-dimensional space. In aerial photo- This case study has the following features, which may be grammetry, overlapping photographs are taken from multiple widespread in underwater object analysis. It involved the positions in three-dimensional space, which is heavily used search for debris over a wide area in water too deep for div- in scientific and research applications as well. The fieldwork ers, so an ROV was employed. The ROV made an erratic search needed to construct a three-dimensional representation of a site pattern as it followed the contours of the steep underwater land- can be dramatically decreased by using digital photogrammetry. scape, the slope of a Pacific atoll. As the ROV pilot examined po- A single camera can be used to capture photographs to be pro- tential pieces of debris, there was no scale marker in the video, cessed off site. The methodology can be seen in the excavation and the debris were far enough apart that they never appeared of the Phanagoria wreck (Zhukovsky et al., 2013). At sites where together in the same frame of the video. Due to the depth, light the use of photogrammetry has been predetermined, a strategy from the surface was completely attenuated, and a single light to overlap photographs in regular patterns can be implemented source originating from the ROV was used. Bubbles caused by (Drap et al., 2007; Canciani et al., 2003).