A Statistical Examination of Image Stitching Software Packages For Use With Unmanned Aerial Systems John W. Gross and Benjamin W. Heumann Abstract There is growing demand for the collection of ultra-high spatial orthomosaic. This issue is typically addressed using aerial resolution imagery, such as is collected using unmanned aerial photogrammetric techniques. systems (UAS). Traditional methods of aerial photogrammetry One of the more conventional ways to handle the creation are often difficult or time consuming to utilize due to the lack of orthomosaics in aerial photogrammetry is through the of sufficiently accurate ancillary information. The goal of this use of automatic aerial triangulation (AAT) and bundle block study was to compare geometric accuracy, visual quality, and adjustment (BBA). In this method, software is able to utilize price of three commonly available mosaicking software pack- interior orientation (IO) information provided by the camera, a ages which offer a highly automated alternative to traditional global positioning system (GPS), and an inertial measurement methods: Photoscan Pro, Pix4D Pro Mapper, and Microsoft Im- unit (IMU) to match individual images together then adjust age Composite Editor (ICE). A total of 223 images with a spatial those blocks of images to match the real world (for a more resolution of 1.26 cm were collected by a UAS along with 70 thorough review of AAT and BBA readers should refer to Wolf ground control points. Microsoft Image Composite Editor had and Dewitt , 2000). The accuracy and quality of these proce- significantly fewer visual errors (Chi Square, p < .001), but it dures are highly dependent on the ability to provide the soft- had the poorest geometric accuracy with a RMSE of 34.7 cm ware with accurate information (Barazzetti et al., 2010; Turner (Tukey-Kramer, p < 0.05). Photoscan had the most visual errors et al., 2012). This is not a problem when using sophisticated (Chi Square, p < 0.001), and a RMSE of 10.9 cm. Pix4D had survey grade metric cameras for which the IOs have been the best geometric accuracy with a RMSE of 7.7 cm, however accurately calculated, and highly accurate GPS and IMU data this was not found to be statistically different from Photoscan which is typically available on manned aircraft platforms. (Tukey-Kramer, p > 0.05). In terms of price, MicrosoftDelivered Image by IngentaDue to the small size and limited power capacity of many Composite Editor was the least expensiveIP: 192.168.39.211 while Pix4D was On: the Fri, commercially24 Sep 2021 available 18:37:04 sUAS, the weight and size of mounted most expensive, althoughCopyright: specific pricing American varies Societydepending for Photogrammetryon sensors, or any and payload, Remote is severelySensing restricted. This often trans- the type of licensing needed. These results suggest that unless lates into the use of consumer grade digital cameras, GPS, and high geometric accuracy or 3D images are required, ICE is the IMU, which typically have insufficient accuracy for conven- best option for most UAS photogrammetric applications. tional orthomosaic techniques (Laliberte et al., 2007; Laliberte et al., 2008; Turner et al., 2012). There can also be significant variability in rotational and angular camera position, degree Introduction of overlap, and illumination between images (Barazzetti et al., Small unmanned aerial systems (sUAS) are a novel comple- 2010). Such errors make the use of conventional photogram- ment to existing image acquisition platforms. Although they metric techniques difficult, and time consuming, to implement. cannot replace satellites and manned aircraft for all applica- Fortunately, recent advancements in the fields of photo- tions, UAS offer three key benefits. They: (a) represent a cost grammetry and computer vision have produced novel tech- effective option for the acquisition of imagery over study niques which offer the potential to not only handle these areas with limited spatial extents, (b) are capable of collecting issues, but to do so in a highly automated fashion. One of the imagery in regions which have traditionally been too danger- most notable techniques is structure from motion (SfM). SfM is ous or delicate for manned aircraft, and (c) fly at drastically of benefit because it does not requirea priori knowledge of any lower altitudes (less than100 m above ground level), which camera parameters or scene information, which complicates can translate into novel fine spatial resolutions on the order of the traditional methods (Choudhary and Narayanan, 2012; 1 cm, which is difficult if not impossible to replicate on any Westoby et al., 2012). SfM utilizes some form of scale invariant other platform (Anderson and Gaston, 2013). feature transform (SIFT) which uses a difference-of-Gaussian sUAS use has been well documented in numerous applica- function to identify “important” features in each image known tions including biological research, precision agriculture, and as keypoints (Lowe, 2004). These keypoints are then matched archeology (Knoth et al., 2013; Verhoeven 2011; Verhoeven in multiple images based on a minimization of Euclidian 2012; Zaman et al., 2011; Zhenkun et al., 2013). This growth distance function (Lindeberg, 2012). By tracking the keypoints in sUAS use will likely continue as government regulations from image to image, SfM is also able to accurately estimate a and safety practices adapt to meet application demands number of external camera parameters such as camera orienta- (Zweig et al., 2015). The use of such miniaturized platforms tion (Westoby et al., 2012). From this combination of infor- and sensors, however, leads to a number of challenges that mation the software is then able to project each pixel into an must be addressed, especially with regards to combining large numbers of images (100+) into a meaningful representa- Photogrammetric Engineering & Remote Sensing tion of the Earth’s surface, a process known as creating an Vol. 82, No. 6, June 2016, pp. 419–425. 0099-1112/16/419–425 Center for Geographic Information Science and Geography © 2016 American Society for Photogrammetry Department, Central Michigan University, Mt. Pleasant, MI and Remote Sensing 48859 ([email protected]). doi: 10.14358/PERS.82.6.419 PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING June 2016 419 05-16 June Peer Reviewed.indd 419 5/19/2016 5:30:00 PM accurate orthomosaic. These keypoints can also be viewed as a Pro, Pix4D, and ICE. The resulting mosaics were then compared dense point cloud which can be converted into a 3D represen- statistically based on geometric accuracy and visual quality. tation of the Earth’s surface, such as a digital surface model. Previous research has utilized a variety of homemade SfM Photoscan Pro scripts to varying degrees of success. For example, Laliberte et Created by the Russian-based company Agisoft in 2010, al. (2008) flew imagery over the Jornada Experimental Range in Photoscan Pro is a 3D model and image stitching software package. It utilizes an adapted form of SfM technology known southern New Mexico and used Autopano pro, a SIFT based soft- ware, to generate keypoints. These keypoints were included in a as the SIFT, proposed by Lowe (2004). At its core, this process custom script known as PreSync which also incorporated a 1 m uses feature points, which are simply geometrically similar digital orthoquad and a 10 m digital elevation model to adjust and distinct regions in an image, e.g., building corners or the top of light posts. These points are then tracked across and improve existing external orientation parameters (EOs). The multiple images creating a series of connections between original images as well as the updated EOs were then put into Leica Photogrammetric Suite to generate the mosaic. They were photographs (Verhoeven, 2011). In addition, this algorithm al- lows Photoscan Pro to automatically and accurately estimate able to obtain an overall RMSE of 47.9 cm which in part was due a large number of internal and external camera parameters to a lack of differential correction on their GPS unit. Turner et al. which previously had to be known and entered manually. (2012) created an automated technique using SIFT and SfM tech- niques to automate the mosaicking of imagery collected over Utilizing such algorithms, software packages such as Pho- two sites of an Antarctic moss bed. They were able to achieve toscan Pro are capable of matching images at the subpixel mean absolute total errors ranging from .103 m to 1.247 m. level (Woodget et al., 2014). In addition, Photoscan Pro is capable of utilizing GCP information directly in the program to A number of commercial SfM software packages have be- come available since 2010. Two of the most popular include increase geometric accuracy. Photoscan Professional (Photoscan Pro), and Pix4D Mapper The Photoscan Pro workflow can be broken down into four Pro (Pix4D). Both of these software have been successfully basic steps: image alignment, dense point cloud formation, used in current research (Vallet et al., 2011; Kung et al., mesh creation, and texture creation. Each of these steps is 2011a; Kung et al., 2011b; Verhoeven, 2011; Verhoeven et al., run independent of each other, and aside from the inclusion 2012; Woodget et al., 2014). of ground control points, they can be run with little to no The goal of this research was to compare commercially avail- user input. It is also important to note that these stages are all independent and can be saved separately for later use or able software packages for use with imagery acquired by sUAS. Specifically, this paper examines geometric accuracy, visual revision. All four steps can be run in a batch process, if the quality, and price as important factors for selecting software to parameters are known in advance. The use of batch process- ing decreases processing time, as well as, allows the software process digital aerial photographs from sUAS. Two commercially to be processed overnight without requiring user intervention.