sensors
Article Using UAV Photogrammetry to Analyse Changes in the Coastal Zone Based on the Sopot Tombolo (Salient) Measurement Project
Pawel Burdziakowski 1,* , Cezary Specht 2 , Pawel S. Dabrowski 2 , Mariusz Specht 3 , Oktawia Lewicka 2 and Artur Makar 4 1 Department of Geodesy, Faculty of Civil and Environmental Engineering, Gdansk University of Technology, Narutowicza 11–12, 80-233 Gdansk, Poland 2 Department of Geodesy and Oceanography, Gdynia Maritime University, 81-347 Gdynia, Poland; [email protected] (C.S.); [email protected] (P.S.D.); [email protected] (O.L.) 3 Department of Transport and Logistics, Gdynia Maritime University, 81-225 Gdynia, Poland; [email protected] 4 Department of Navigation and Hydrography, Polish Naval Academy, Smidowicza 69, 81-127 Gdynia, Poland; [email protected] * Correspondence: [email protected]
Received: 16 June 2020; Accepted: 16 July 2020; Published: 18 July 2020
Abstract: The main factors influencing the shape of the beach, shoreline and seabed include undulation, wind and coastal currents. These phenomena cause continuous and multidimensional changes in the shape of the seabed and the Earth’s surface, and when they occur in an area of intense human activity, they should be constantly monitored. In 2018 and 2019, several measurement campaigns took place in the littoral zone in Sopot, related to the intensive uplift of the seabed and beach caused by the tombolo phenomenon. In this research, a unique combination of bathymetric data obtained from an unmanned surface vessel, photogrammetric data obtained from unmanned aerial vehicles and ground laser scanning were used, along with geodetic data from precision measurements with receivers of global satellite navigation systems. This paper comprehensively presents photogrammetric measurements made from unmanned aerial vehicles during these campaigns. It describes in detail the problems in reconstruction within the water areas, analyses the accuracy of various photogrammetric measurement techniques, proposes a statistical method of data filtration and presents the changes that occurred within the studies area. The work ends with an interpretation of the causes of changes in the land part of the littoral zone and a summary of the obtained results.
Keywords: photogrammetry; UAV; coastal line; tombolo; salient; Sopot
1. Introduction Unmanned platforms (water, land and air) are devices capable of moving in a specific environment without the presence of an operator on board [1,2]. Their movement can be remotely controlled by a human being or programmed and executed automatically. Their main advantage is their ability to perform their task in areas where a manned mission would be difficult or impossible. Researchers and engineers soon noted these properties and started to use unmanned vehicles as mobile platforms for research equipment [3–9]. As a result, it enabled research in new locations and with unprecedented frequency. The coastal zone is a transition zone between the land and the water environment [10]. The analysis of changes taking place in this zone requires the application of various measurement methods and techniques capable of drafting three-dimensional environmental models [11,12]. Techniques of
Sensors 2020, 20, 4000; doi:10.3390/s20144000 www.mdpi.com/journal/sensors Sensors 2020, 20, 4000 2 of 21
terrestrial, airborne or mobile laser scanning and satellite, aerial or low ceiling photogrammetry are currently used to record the shape of the land surface [6,7,10,12–24]. Low-cell photogrammetry has become a very good source of morphological data in the coastal zone. The paper [25] compares the differences between the anthropogenic and the natural coastal zone, based on the morphological reconstruction of dunes, using a low-cost unmanned aerial vehicle (UAV) and the photogrammetric reconstruction method, in which the authors were able to provide high resolution digital surface model (DSM). In the study [26], the researchers examined the applicability of UAVs and structure from motion (SfM) algorithms to reconstruct the costal environment. Authors compared the models from terrestrial laser scanning (TLS) and UAVs, and then generated high resolution DSM form the combined data. Single or multibeam echo-sounders are used for recording the shape of the bottom surface of a body of water [27–30]. Multibeam echo-sounders (MBES), which were originally designed for deep-water measurements, are commonly used to obtain high-resolution bathymetric data in coastal areas [28,30,31]. Recording the shape of both surfaces at specific time intervals and comparing the generated spatial models allows one to analyse the time changes that occurred in the environment and determining their dynamic characteristics [32,33]. The main factors influencing the shape of the beach, shoreline and seabed include undulation, wind and coastal currents [34,35]. Beaches are constantly transformed by waves and wind, and the material forming them is subject to constant movement from the sea to land and back [10,36]. Incoming waves cause an ascending movement, whereas return flow causes a descending movement. If the waves hit the shore at a certain angle, then the movement to and from the shore overlaps with the movement of grains along the coast. In addition, in the moderate climate zone, winter is characterized by frequent storms. Therefore, shore erosion dominates during this period, whereas in summer, shore deposition prevails. Additionally, these changes heavily depend on human activity in the coastal zone involving consisting of the construction of infrastructure and other facilities [19,37]. Coastal currents in this area that displace bottom sediments are responsible for creating links between the mainland and coastal islands or hydro-technical structures. These forms are called salient or tombolo [38–43]. Tombolo or salient is an accumulative form of coastal relief closely related to the influence of coastal currents. The littoral transportation is decreased due to the attenuated wave and longshore currents in the area sheltered by the breakwater. The material is deposited and it gradually reaches more and more towards the island or structure (pier or breakwater). Depending on the conditions, the trapping sand will develop into a tombolo or salient. If the length of the pier is equal to or longer than 0.8 times the distance between the shore and the breakwater, tombolo will be formed. Here, in the case of the Sopot pier, the breakwater length and its distance do the shoreline ratio is equal to 0.78, hence it is likely that a tombolo will eventually develop. Furthermore, the desire to form a tombolo in this area is constantly stopped by active human activity through regular removal of accumulated deposits by machines (excavators, etc.). At this stage, the presented form is the salient; however, parameters other than the breakwater length and distance may influence the accumulation pattern. Apart from that, these phenomena cause continuous and multidimensional changes in the shape of the seabed and the Earth’s surface, and when they occur in an area of intense human activity, they should be constantly monitored [44]. Tombolo or salient is a local phenomenon that allows for use of unmanned platforms with measuring equipment for testing. Sopot, a city located in northern Poland on the Gdansk Bay, is a popular tourist resort (Figure1).
Downloaded from mostwiedzy.pl In the coastal zone of the city, there is a platform perpendicular to the shoreline, adapted to serve sports vessels and small passenger ships. In this area, tombolo, salient and changes in land structure influence human activity. A shoal patch in the area, due to sediment transport, poses a threat to the traffic of ships and tourist vessels. The movement of sand on the beach makes it necessary to keep it in a condition suitable for tourists. Periodic analyses and identification of processes that take place in this area allow for accurate planning of anthropopressure [44]. For this purpose, we developed and implemented a complementary methodology of evaluation of the phenomena occurring within the littoral area using unmanned platforms [42]. Sensors 2020, 20, 4000 3 of 21 Sensors 2020, 20, x FOR PEER REVIEW 3 of 20
Figure 1. Study area-location: city Sopot, Poland. Figure 1. Study area-location: city Sopot, Poland. The research [42] presents the genesis of tombolo (salient) formation in Sopot in great detail and describesThe research the methodology[42] presents the of integratedgenesis of spatialtombolo measurements (salient) formation of this in phenomenon, Sopot in great and detail finally, and itdescribes proves the the accuracy methodology of measurement of integrated techniques spatial measurements used. This study of [ 42this] uses phenomenon, a unique combination and finally, of it bathymetricproves the accuracy data obtained of measurement from a hydrographic techniques motorboatused. This study to the [42] 0.6 m uses isobath, a unique unmanned combination surface of vesselbathymetric (USV) data to the obtained 0.2 m isobath, from a photogrammetrichydrographic motorboat data obtained to the 0. from6 m isobath, unmanned unmanned aerial vehicles surface (UAVs)vessel (USV) and terrestrial to the 0.2 laser m isobath, scanning photogrammetric (TLS), and geodetic data dataobtained from from precision unmanned measurements aerial vehicles with receivers(UAVs) and of global terrestrial navigation laser satellitescanning systems (TLS), (GNSS).and geodetic This greatdata varietyfrom precision of different measurements sources of spatial with informationreceivers of allows global one navigation to gain very satellite detailed systems knowledge (GNSS). about This changes great taking variety place of different in the environment. sources of Onspatial the information other hand, allows it entails one several to gain issues very relateddetailed to knowledge the interoperability about changes of spatial taking data place sets in and the theirenvironment. harmonisation. On the other hand, it entails several issues related to the interoperability of spatial data sets andThis their paper harmonisation. focuses on photogrammetric measurements from unmanned aerial vehicles for the measurementThis paper focuses campaign on photogrammetric of the unique tombolomeasurements (salient) from phenomenon unmanned aerial in the vehicles studied for area the (Figuremeasurement1). The articlecampaign discusses of the theunique procedure tombolo of developing(salient) phenomenon photogrammetric in the datastudied from area two (Figure di fferent 1). measurementsThe article discusses carried out the by procedure unmanned of aerial developing vehicles. photogrammetric Measurements using data UAV from photogrammetry two different Downloaded from mostwiedzy.pl weremeasurements aimed at carried measuring out by morphological unmanned aerial changes vehicles. occurring Measurements within the using beach. UAV Within photogrammetry the beach, twowere oppositely aimed at measuring motivated pressuresmorphological are stumbling. changes occurring The environmental within the forcesbeach. that Within naturally the beach, form two the surfaceoppositely of the motivated beach and pressures the human are pressure stumbling. that Thecontinuously environmental adjust forcesthe area that to the naturally requirements form the of thesurface touristic of the activity. beach and In order the human to reveal pressure changes that within continuously the beach adjust surface, the area the prepared to the requirements point clouds of werethe touristic initially activity. filtered withIn order statistical to reveal methods. changes This within operation the beach minimized surface, outlying the prepared points andpoint allowed clouds towere precisely initially align filtered the with point statistical clouds. Asmethods. a consequence, This operation the changes minimized were outlying calculated points using and distances allowed directlyto precisely between align twothe point clouds. As Detailed a consequence, elements the of thechanges procedure were calculated have been using described distances and presenteddirectly between in the article. two point The cloupublicationds. Detailed has been elements divided of the into procedure six sections. have The been first described section is and the presented in the article. The publication has been divided into six sections. The first section is the Introduction, which presents the motivation for this study. The second section, Materials and
Sensors 2020, 20, 4000 4 of 21
Introduction, which presents the motivation for this study. The second section, Materials and Methods, describes the tools and methods used to process the data. The third section discusses the results obtained. The paper ends with a Conclusions section which summarizes the most important aspects of the study.
2. Materials and Methods Tombolo (salient) measurement campaigns in Sopot spanned three years. The most important measurements realised with a full range of measurement methods and the use of unmanned systems, took place in November 2018 and November 2019 [42]. The article covers this two-year period, where measurements were made with a full range of unmanned methods and in accordance with the developed methodology presented in [42]. The dynamic development of UAV technology during this period resulted in the use of various unmanned platforms equipped with various cameras.
2.1. Data Acquisition Process A photogrammetric flight was performed in 2018 with a type DJI Mavic Pro UAV, while in 2019, a DJI Mavic Pro 2 was used for image acquisition. Aircraft of this type are commercial flying platforms designed and intended mainly for recreational flights and amateur filmmakers. The photogrammetric community soon appreciated the versatility and reliability of these devices. They gained popularity mainly due to their simplicity and intuitive software. The technical data of both platforms are presented in Table1.
Table 1. Technical data of DJI Mavic Pro and DJI Mavic Pro 2 unmanned aerial vehicles.
Technical Data DJI Mavic PRO DJI Mavic PRO 2 Dimensions (L W H) (mm) 305 244 85 322 242 84 × × × × × × Weight (g) 734 907 Maximum rising speed (m/s) 5 5 Maximum ascending velocity (m/s) 3 3 Maximum advance velocity (km/h) 65 72 Maximum altitude (m) 5000 6000 Maximum flight time (min) 27 31 Maximum hovering time (min) 24 29 Mean flight time (min) 21 25 Maximum flight range (km) 13 18 Permissible operating temperature range ( C) 0 to 40 10 to 40 ◦ − Satellite Navigation Systems GPS/GLONASS GPS/GLONASS
Unmanned aerial vehicles (UAVs) are equipped with stabilized visible light cameras of type F230 and L1D-20C with 12 and 20 million pixels, respectively. The technical data of both cameras are presented in Table2. In terms of photogrammetry, the coastal zone combines a land rich in solid textures with water that is extremely variable in terms of images and very luminous. In this case, the UAV measurement focused on the beach, which was considered a priority area, with the water area and the offshore pier being treated as auxiliary areas. It was assumed that a ground sampling distance (GSD) of
Downloaded from mostwiedzy.pl approximately 2 cm/pixel would be sufficient to generate a numerical model of the terrain and point clouds of the land surface, and it would allow for further analysis of the phenomena with satisfactory accuracy. Based on this value, the height of the flight over the beach was determined. No required minimum GSD values for the auxiliary area were assumed. Sensors 2020, 20, 4000 5 of 21
The data of the planned flight patterns are presented in detail in Table3. Because the parameters of the cameras used in both campaigns differ (Table2) for a fixed GSD, the average flight altitude (hMAGL) over the priority area was calculated using the formula:
IW GSD FR hMAGL = (1) 100 SW
where, IW—image width expressed in pixels (px), GSD—the given ground sampling distance in pixels per centimetre (px/cm), FR—actual focal length of the camera (mm) and SW—actual sensor width (mm).
Table 2. Data of FC220 and L1D-20c (Hasselblad) cameras.
Technical Data F230 L1D-20c (Hasselblad) Sensor size 1/2.3”, 12.35 MP 1”, 20 MP Pixel size (µm) 1.55 2.41 1 1 Lenses (Field of vision—FOV) FOV 78.8◦ (28 mm ) f/2.2 FOV 77◦ (28 mm ) f/2.2 Focus from 0.5 m to , auto/manual focus from 1 m to , auto/manual focus ∞ ∞ 100–6400 (video), 100–12,800 ISO sensitivity range 100–3200 (video), 100–1600 (photo) (photo) Electronic shutter time 8 s–1/8000 s 8 s–1/8000 s Image size (pixel) 4000 3000 5472 3648 × × Single shot, Burst shooting: Single shot, Burst shooting: 3/5/7 frames, 3/5 frames, Auto Exposure Photo modes Auto Exposure Bracketing (AEB): 3/5 Bracketing (AEB): 3/5 bracketed bracketed frames at 0.7 EV, Interval frames at 0.7 EV, Interval C4K: 4096 2160, 24 fps 4K: 3840 2160 24/25/30 p × × 4K: 3840 2160, 24/25/30 fps 2.7K: 2688 1512 × × Video modes 2.7K: 2720 1530, 24/25/30 fps 24/25/30/48/50/60 p × FHD: 1920 1080, 24/25/30/48/50/60/96 fps FHD: 1920 1080 × × HD: 1280 720, 24/25/30/48/50/60/120 fps 24/25/30/48/50/60/120 p × Image file format JPEG, DNG MP4/MOV (MPEG-4 AVC/H.264, Video file format MP4, MOV (MPEG-4 AVC/H.264) HEVC/H.265) 1 35 mm format equivalent.
Flights performed in the 2018 campaign consisted of two different plans. The first plan, based on the double grid scheme [45], was realized in automatic mode over the priority area. This scheme is mainly used for modelling urban areas, where it is important to obtain information on the faces of buildings or areas with very variable relief. The land area was covered twice with a demanding and relatively long flight. Such a way of taking pictures minimizes information loss, but the flight consumes more energy of the main UAV battery and takes longer. The second scheme was implemented in manual mode. In this case, the operator manually controlled the UAV over the pier structure and over the sea area (Figure2a). As it was assumed, the additional area, relevant for the model as a whole, did not require modelling with a pre-determined minimum ground sampling distance (GSD). The absence of this limitation allowed for a significant increase in flight altitude and, as a consequence, obtaining a larger sea surface image visible on the orthophotomap. Downloaded from mostwiedzy.pl SensorsSensors 20202020, 20, ,20 x ,FOR 4000 PEER REVIEW 6 of6 20 of 21
(a) (b)
FigureFigure 2. 2.LocationLocation of ofcameras cameras (black (black spots) spots) and and the the implemented implemented flight flight plan plan (a) (flighta) flight 2018 2018 (b) ( bflight) flight 2019. 2019.
TheThe photogrammetric photogrammetric flight flight in inthe the 2019 2019 campaign campaign was was planned planned and and executed executed according according to tothe the singlesingle grid grid plan plan [45]. [45 Due]. Due to the to thesize sizeof the of area the areaand andthe wind the wind blowing blowing at the at speed the speed of over of 5 over m/s, 5the m /s, plannedthe planned flight flighttime exceeded time exceeded the maximum the maximum safe time safe timefor the for unmanned the unmanned aircraft aircraft used. used. Therefore, Therefore, it wasit wasdecided decided to divide to divide the theresearch research area area into into three three smaller smaller ranges. ranges. This This enabled enabled making making three three safe safe flightsflights lasting lasting 18 18 min min and and to to completely completely cover the areaarea underunder study.study. In In this this case, case, the the measurement measurement was wasperformed performed at aat fixed a fixed altitude, altitude, which which did did not not allow allow for for depicting depicting a sea a sea area area as largeas large as inas thein the first first case. case. When analysing the results, note that no tie points can be found on luminous and non-textured surfacesWhen [analysing46]. Examples the results, of such note surfaces that no include tie points water, can snow,be found glass on walls luminous of high-rise and non buildings-textured or surfaceswindows. [46]. ThisExamples makes of it such impossible surfaces in include practice water, to generate snow, glass a point walls cloud of high or orthophoto-rise buildings of these or windows.surfaces [ This47–50 makes]. In such it impossible cases, an increase in practice in flight to generate altitude is a applied point cloud which, or in orthophoto turn, allows of to these extend surfacesthe terrain [47– size50]. In of such the photo cases, and an illustrateincrease in a largerflight altitude terrain context. is applied Then, which, at the in expenseturn, allows of loss to inextend visible thetexture terrain details size of and the geometricphoto and qualityillustrate of a the larger model, terrain it is context. possible Then, to develop at the expense a photogrammetric of loss in visibleproduct texture [51 ].details Such and recommendations geometric quality are of the included model, init is the possible Pix4D to softwaredevelop a documentation photogrammetric [52 ]. productThis method [51]. Such was recommendations used for the 2018 campaign.are included The in auxiliarythe Pix4D area software was covered documentation with photos [52]. taken This at methodaltitudes was up used to 260 for m. the 2018 campaign. The auxiliary area was covered with photos taken at altitudes up to 260 m. Table 3. Flight details. Table 3. Flight details. 2018 Priority Area 2018 Auxiliary Area 2019 Flight path2018 Double Priority grid Area 2018 Free Auxiliary flight Area Single2019 grid Ground sampling distance (GSD) 2.25 8.4Single 2.21 Flight path Double grid Free flight Number of photos taken 621 413 462grid GroundCoverage sampling (longitudinal distance/traverse) (GSD) (%) 802.25/80 85–95/85–958.4 652.21/65 Flight altitude above ground level (AGL) 60 150–260 100 Number of photos taken 621 413 462 Coverage (longitudinal/traverse) (%) 80/80 85–95/85–95 65/65 2.2.Flight Processing altitude of above Photogrammetric ground level Data 60 150–260 100 Downloaded from mostwiedzy.pl The result-processing(AGL) stage starts with importing all of the photos to computer software and entering the processing settings. Different commercial photogrammetric software was used for both 2.2.cases, Processing Pix4D of Mapper Photogrammetric and Agisoft Data Metashape, respectively. The project calculation for both cases was doneThe di resultfferently-processing with varied stage user starts access with to theimporting initial processing all of the settings.photos to Each computer manufacturer software also and uses enteringits own, the di ffprocessingerent file formats, settings. which Different are notcommercial mutually photogrammetric compatible. This software implies the was necessity used for to both make cases,data Pix4D processing Mapper and and exchange Agisoft uniform. Metashape, respectively. The project calculation for both cases was done differently with varied user access to the initial processing settings. Each manufacturer also uses its own, different file formats, which are not mutually compatible. This implies the necessity to make data processing and exchange uniform.
SensorsSensors2020 2020, 20, 20, 4000, x FOR PEER REVIEW 77 of of 20 21
uses its own, different file formats, which are not mutually compatible. This implies the necessity to makeFigure data3 processi presentsng theand photogrammetricexchange uniform. process used to study the presented phenomenon. An analogueFigure algorithm3 presents wasthe photogrammetric used in the studies process [53], which used modelledto study the the presented topography phenomenon. of a quarry withAn theanalogue following algorithm initial rules was assumed:used in the studies [53], which modelled the topography of a quarry with the following initial rules assumed: 1. application of georeferencing directly and/or through the use of ground control points, 1. standardapplication processing of georeferencing settings (as directly proposed and/or by through the software); the use of ground control points, standard 2. exportingprocessing results settings in the(as formproposed of a high-densityby the software); point cloud to LAS format, a surface model to OBJ 2. formatexporting and results a numerical in the coverage form of a model high-density and orthophotomap point cloud to to LAS TIFF format, format; a surface model to 3. theOBJ UAV format navigation and a numer systemical records coverage camera model position and orthophotomap relative to the WGS84 to TIFF ellipsoid format; and represents 3. itthe with UAV geodetic navigation coordinates systemB, L recordsand h (latitude, camera position longitude, relative ellipsoidal to the height); WGS84 ellipsoid and represents it with geodetic coordinates B, L and h (latitude, longitude, ellipsoidal height); 4. position of the photogrammetric warp points is expressed in coordinates in the Polish PL-2000 4. position of the photogrammetric warp points is expressed in coordinates in the Polish PL-2000 system of flat coordinates, and their altitude is expressed relative to the quasigeoid in the Polish system of flat coordinates, and their altitude is expressed relative to the quasigeoid in the Polish PL-EVRF2007-NH altitude system; PL-EVRF2007-NH altitude system; 5. the location of ground control points is measured with an accurate method of differential satellite 5. the location of ground control points is measured with an accurate method of differential positioning GNSS RTK (accuracy 2 cm, p = 0.95); satellite positioning GNSS RTK (accuracy 2 cm, p = 0.95); 6.6. thethe Polish Polish PL-2000 PL-2000 flat flat coordinate coordinate systemsystem isis thethe targettarget coordinatecoordinate systemsystem of the study; 7.7. altitudesaltitudes are are related related to to the the quasigeoid quasigeoid PL-EVRF2007-NH.PL-EVRF2007-NH.
Data processing: 2018 - Photo alignment - GCP localization Data acquisition: - Camera alignment optimization - Flight plan Exporting - Dense point cloud building - Camera parameters results - Mesh building - Direct georeferencing - Texture building - DEM building - Orthomosaic building Data analysis: - Importing point clouds - Clouds alIgnment - Manual segmentation - Filtering Data processing: - Clouds comparing - Photo alignment - Exporting results Data acquisition: 2019 - GCP localization - Flight plan - Camera alignment optimization - Camera parameters Exporting - Dense point cloud building - Direct georeferencing results - Mesh building - Ground Control Points - Texture building - RTK Measurements - DEM building - Orthomosaic building Legend:
- Pix4Dmapper Pro version 4.2.26
- Agisoft Metashape version 1.5.0
- CloudCompare version 2.11 beta
FigureFigure 3.3. DataData processing.
TheThe results results of handling of handling data data from fromphotogrammetric photogrammetric measurements measurements of the 2018 of the and 2018 2019 campaigns and 2019 incampaigns the form of in orthophotomaps the form of orthophotomaps are presented are in Figures presented4a and in 5Figurea, respectively. 4a and Figure Numerical 5a, respectively. land cover Downloaded from mostwiedzy.pl modelsNumerical are presented land cover in models Figures are4b andpresented5b, respectively. in Figure In4b addition, and Figure high-density 5b, respectively. point cloudsIn addition, were generatedhigh-density for furtherpoint clouds comparative were generated analyses. for further comparative analyses.
Sensors 2020, 20, 4000 8 of 21 Sensors 2020, 20, x FOR PEER REVIEW 8 of 20
Sensors 2020, 20, x FOR PEER REVIEW 8 of 20
(a) (b) (a) (b) FigureFigure 4. Orthophotomap 4. Orthophotomap (a) and ( aDSM) and (b DSM) of the (b area) of adjacent the area to adjacent the pier in to Sopot, the pier measurement in Sopot, of measurement 2018. Figureof 2018. 4. Orthophotomap (a) and DSM (b) of the area adjacent to the pier in Sopot, measurement of 2018.
(a) (b) Figure 5. Orthophotomap(a )( a) and DSM (b) of the area adjacent to the pier in Sopot, measurement(b) of 2019.
FigureFigure 5. Orthophotomap 5. (a) anda DSM (b) of theb area adjacent to the pier in Sopot, measurement of 2019. 2.3. Accuracy CharacteristicsOrthophotomap of Photogrammetric ( ) and DSM Studies ( ) of the area adjacent to the pier in Sopot, measurement of 2019. 2.3. AccuracyDetails on Characteristics measurement of conditionsPhotogrammetric in 2018 Studies and 2019 have been compared in Table 4. Please note 2.3. Accuracy Characteristics of Photogrammetric Studies that theDetails measurement on measurement accuracy conditions reports generated in 2018 and by 2019 Pix4D have and been Agisoft compared Methashape in Table software 4. Please differ note inthat terms theDetails me of asurement content on measurement and accuracy data presentation. reports conditions generated Forin 2018 this by and study,Pix4D 2019 and the have valuesAgisoft been givenMethashape compared in Table in software Table4 have4. differ Pleasebeen note recalculated and presented in uniform units to enable comparative analysis. in that terms the of measurement content and data accuracy presentation. reports generated For this study, by Pix4D the values and Agisoft given Methashapein Table 4 have software been differ In both measurement campaigns, the UAVs used could record images with metadata on the recalculatedin terms of and content presented and in data uniform presentation. units to enable For thiscomparative study, the analysis values. given in Table4 have been current UAV position. These data are saved in EXIF (exchangeable image file format). The current recalculatedIn both measurement and presented campaigns, in uniform the units UAVs to used enable could comparative record images analysis. with metadata on the position, recorded by an onboard GPS receiver and saved in the image metadata, can be compared currentIn UAV both position. measurement These data campaigns, are saved the in EXIF UAVs (exchangeable used could recordimage file images format). with The metadata current on the with the external orientation elements (EOP) determined at the aero-triangulation stage. In this way, position,current recorded UAV position. by an onboard These data GPS arereceiver saved and in EXIFsaved (exchangeablein the image me imagetadata, file can format). be compared The current Downloaded from mostwiedzy.pl for each image, the absolute position error of the central projection position was determined in meters withposition, the external recorded orientation by an onboard elements GPS (EOP) receiver determined and saved at the in aero the image-triangulation metadata, stage. can In be this compared way, with andfor each the standardimage, the deviation absolute positionof the position error of error the central for the projection entire block position of images was determined was calculated. in meters The the external orientation elements (EOP) determined at the aero-triangulation stage. In this way, for each 2018and thedata standard presented deviation in Table of 4 the indicate position low error error for values the entire in the block horizontal of images plane was (x, calculated.y) and vertical The image, the absolute position error of the central projection position was determined in meters and the plane2018 data (z) and presented low standard in Table deviation 4 indicate (σ), low which error proves values that in positionthe horizontal measurements plane (x, are y) andvery vertical stable. standard deviation of the position error for the entire block of images was calculated. The 2018 data Onplane this (z )basis, and lowit is standardconcluded deviation that georeference (σ), which for proves each thatimage position was determined measurements correctly, are very with stable. high presented in Table4 indicate low error values in the horizontal plane ( x, y) and vertical plane (z) and accuracyOn this basis, and there it is concludedwere no significant that georeference deviations. for The each values image wer wase within determined the accuracy correctly, range with typical high σ foraccuracylow single standard -andfrequency there deviation were GPS no receivers (significant), which [54– provesdeviations.58]. that The position values measurements were within the are accuracy very stable. range typical On this basis, forit single is concluded-frequency that GPS georeference receivers [54 for– each58]. image was determined correctly, with high accuracy and there
Sensors 2020, 20, x FOR PEER REVIEW 9 of 20
Table 4. Data obtained from reports generated by photogrammetric software.
Parameter 2018 2019 Ground control points (GCP) No Yes Total number of images with georeferencing 1037 462 Sensors 2020, 20, 4000 9 of 21 Number of images used for modelling 964 233 GCP measurement accuracy NA * RTK GPS were no significantMean deviations. reprojection The error values (pix) were within the accuracy0.301 range typical for0.514 single-frequency GPS receiversTotal number [54–58 ].of TPs connection points (3D) 1,454,125 218,226 Median of key points per image 19,475 40,000 TableDirect 4. Data georeference obtained from reports generated by photogrammetricGPS software.GPS MedianParameter matches per image 20184717.91 4000 2019 Mean absolute camera position error (x,y,x) (m) 0.165, 0.167, 0.272 0.597, 2.205, 65.441 Ground control points (GCP) No Yes MeanTotal camera number position of images standard with georeferencing deviation (x,y,z) σ 0.052, 1037 0.059, 0.067 0.020, 0.018, 462 0.014 Number of of images points used in dense for modelling point cloud 21,765 964,551 116,831 233,423 * DG:GCP double measurement grid, SG:accuracy single grid, Free: manually operated NA * flight, NA: not available. RTK GPS Mean reprojection error (pix) 0.301 0.514 ExternalTotal number orientation of TPs elements connection specified points (3D) for the 2019 flight 1,454,125 have a larger mean 218,226absolute camera Median of key points per image 19,475 40,000 position error. These values result from the use of ground control points in the photogrammetric Direct georeference GPS GPS process and determiningMedian matches their per position image in relation to another 4717.91 reference system. The 4000 measurement performedMean by absolute an on camera-board positionGPS records error (x,y,x)the position (m) and 0.165, altitude 0.167, relative 0.272 to the 0.597,WGS84 2.205, ellipsoid. 65.441 The GCP Meanposition camera was position determined standard relative deviation to the (x,y,z) Polishσ PL0.052,-2000 0.059,flat coordinate 0.067 system 0.020, and 0.018, the 0.014 altitude relative toNumber PL-EVRF2007 of points-N inH dense quasigeoid. point cloud As presented in Table 21,765,551 4, the error values 116,831,423 in the horizontal plane (x, y) are within* DG: double the range grid, SG: typical single grid,for Free:GPS manuallyreceivers operated used flight,in commercial NA: not available. UAVs. The absolute error in the vertical plane (z) is significantly greater (65.4 m) and results directly from using different External orientation elements specified for the 2019 flight have a larger mean absolute camera altitude reference systems. The determined ellipse errors for EOP are shown in Figure 6b. For DJI position error. These values result from the use of ground control points in the photogrammetric Mavic Pro 2, a significantly smaller standard deviation of the recorded photo position was observed. process and determining their position in relation to another reference system. The measurement The photogrammetric warp points were distributed evenly over the entire area of the study and performed by an on-board GPS records the position and altitude relative to the WGS84 ellipsoid. their position was measured with an accurate GNSS RTK satellite positioning method. Five points The GCP position was determined relative to the Polish PL-2000 flat coordinate system and the altitude were located on stable infrastructure elements, such as concrete sidewalks running along the beach. relative to PL-EVRF2007-NH quasigeoid. As presented in Table4, the error values in the horizontal Two more were placed on the concrete marina breakwater constituting a part of the building. Table plane (x, y) are within the range typical for GPS receivers used in commercial UAVs. The absolute 5 presents the roots of the mean square error (RMSE) for the location of checkpoints. Figure 6a is a error in the vertical plane (z) is significantly greater (65.4 m) and results directly from using different graphical representation of the errors for individual checkpoints. The shape and colour of the altitude reference systems. The determined ellipse errors for EOP are shown in Figure6b. For DJI respective ellipses represent the distribution of the GCP location error. Mavic Pro 2, a significantly smaller standard deviation of the recorded photo position was observed.
(a) (b) Downloaded from mostwiedzy.pl FigureFigure 6. 6. EllipsesEllipses of of GCP GCP (a (a),), and and EOP EOP (b (b) )location location errors. errors.
The photogrammetric warp points were distributed evenly over the entire area of the study and their position was measured with an accurate GNSS RTK satellite positioning method. Five points were located on stable infrastructure elements, such as concrete sidewalks running along the beach. Two more were placed on the concrete marina breakwater constituting a part of the building. Table5 presents the roots of the mean square error (RMSE) for the location of checkpoints. Figure6a is a Sensors 2020, 20, 4000 10 of 21
graphical representation of the errors for individual checkpoints. The shape and colour of the respective Sensors 2020, 20, x FOR PEER REVIEW 10 of 20 ellipses represent the distribution of the GCP location error. Table 5. Mean square errors in the location of ground control points. Table 5. Mean square errors in the location of ground control points. Number of RMSE X RMSE RMSE Z RMSE XY Total RMSE Number of RMSE X RMSE Y RMSE Z RMSE XY Total RMSE GCP (cm) Y(cm) (cm) (cm) (cm) GCP (cm) (cm) (cm) (cm) (cm) 7 7.02742 4.46724 0.705093 8.32712 8.35692 7 7.02742 4.46724 0.705093 8.32712 8.35692 Internal camera orientation elements were taken from the database of the photogrammetric softwareInternal use camera and optimized orientation using elements autocalibration were taken during from the preliminarydatabase of themodel photogrammetric development softwareprocess. useDetailed and optimized internal orientation using autocalibration element values during after the optimisation preliminary and model standard development deviation process. are Detailedpresented internal in Table orientation 6. element values after optimisation and standard deviation are presented in Table6. Table 6. Internal camera orientation elements. Table 6. Internal camera orientation elements. Camera F (pix) Cx (pix) Cy (pix) K1 K2 K3 P1 P2 CameraF220 F2808.897 (pix) C1956.756x (pix) 1498.347 Cy (pix) 0.040 K1 −0.128K2 0.118K 3 0.000 P1 0.000P 2 F220σ 2808.8970.415 1956.7560.089 1498.3470.082 0.0400 00.128 0 0.1180 0.0000 0.000 − σL10 0.4154256 2691.02 0.089 1803.22 0.082 −0.0182 0 −0.0168 0 0.0125 0 −0.00236 0 −0.00139 0 L10 4256 2691.02 1803.22 0.0182 0.0168 0.0125 0.00236 0.00139 σ 0.401 0.13 0.063 0.000034− 0.00013− 0.00015 0.0000024− 0.0000023− σ 0.401 0.13 0.063 0.000034 0.00013 0.00015 0.0000024 0.0000023 3. Results 3. Results 3.1. Filtration of Point Clouds 3.1. Filtration of Point Clouds It is very difficult to correctly and precisely reconstruct the water surface and it may even prove impossibleIt is very in di practice.fficult to This correctly is because and popular precisely algorithms reconstruct [59 the–62] water used surfacein photogrammetric and it may even software prove impossibledetection of in key practice. points This in such is because areas are popular burdened algorithms with a [59large–62 ]error. used Figure in photogrammetric 7 shows homologous software detectionpoints found of key in pointsa stereo in-pair such depicting areas are the burdened water and with land a area. large They error. clearly Figure illustrate7 shows the homologous problems pointsin the found reconstruction in a stereo-pair of variable depicting surfaces. the waterIn these and surfaces, land area. the Theyalgorithm clearly did illustrate not detect the any problems key inpoints. the reconstruction On the beach, ofthe variable key points surfaces. and matches In these are surfaces, evenly distributed the algorithm within did the not image detect coverage any key points.area. For On clarity,the beach, Figure the key7b shows points images and matches with about are evenly 800 pairs distributed of points. within In this the area, image an coverage image area.showing For clarity, 4000 pairs Figure would7b shows be illegible. images In with such about cases, 800 the pairs technique of points. for increasing In this area, flight an image altitude showing (here 4000used pairs in would2018) allows be illegible. obtaining In such a cases,slight the improvement technique for in increasing the surface flight area altitude of the (here generated used in 2018)orthophotomap. allows obtaining a slight improvement in the surface area of the generated orthophotomap. Downloaded from mostwiedzy.pl
(a) (b)
FigureFigure 7.7. HomologousHomologous points in the the water water ( (aa)) and and land land (b (b) )area. area.
A rectified image may be fitted and will become part of the orthophotomap, provided that homologous points are found for a given stereo-pair. As shown in Figure 7 homologous points are
Sensors 2020, 20, 4000 11 of 21
A rectified image may be fitted and will become part of the orthophotomap, provided that
homologousSensors 2020, 20 points, x FOR PEER are found REVIEW for a given stereo-pair. As shown in Figure7 homologous points are11 of not 20 generated on the water surface. As a result, only photographs with any fixed infrastructural elements maynot begenerated used for on the the construction water surface. of orthophotomap. As a result, only Thus, photographs around the fixedwith objectany fixed it is onlyinfrastructural possible to generateelements an may orthophotomap be used for the up construction to the maximum of orthophotomap. terrain width Thus, of the around photo. the The fixed terrain object size it is ofonly the photopossible is directly to generate proportional an orthophotomap to the flight up altitude, to the maximum according terrain to the followingwidth of th relationship:e photo. The terrain size of the photo is directly proportional to the flight altitude, according to the following relationship: hMAGL LW ℎ푀퐴퐺퐿 = 퐿푊 (2) FR = SW (2) 퐹푅 푆푊 where: hMAGL—flight altitude (m), FR—the actual focal length of the camera (mm), LW—terrain where: ℎ푀퐴퐺퐿 —flight altitude (m), 퐹푅 —the actual focal length of the camera (mm), 퐿푊 —terrain width of the image and SW—the actual width of the sensor (mm). Using this relationship, we can width of the image and 푆푊—the actual width of the sensor (mm). Using this relationship, we can determine which maximum distance between the water area and fixed objects will be depicted on determine which maximum distance between the water area and fixed objects will be depicted on the the orthophotomap. Figure8a presents a fragment of the study with the aero-triangulated images orthophotomap. Figure 8a presents a fragment of the study with the aero-triangulated images and andthe theimage image visible visible on them on them mapped. mapped. The described The described phenomenon phenomenon is schematically is schematically presented presented in Figure in Figure8b, where8b, where green green marks marks the terrain the terrain area area of the of the photo photo that that can can be be included included in in the the orthophotomap, orthophotomap, whereaswhereas red red marks marks the the rejectedrejected photos.photos. FixedFixed infrastructure elements are marked in in orange. orange.
(a) (b)
FigureFigure 8. 8.Generating Generating an an orthophotomap orthophotomap over over the the water water area area (a) during (a) during the flight the in flight 2019, in and 2019, a diagram and a illustratingdiagram illustrating the problem the (problemb). (b).
IncorrectlyIncorrectly located key key points points affect affect the the accuracy accuracy of the of high the- high-densitydensity point pointcloud; cloud;thus, other thus, otherphotogrammetric photogrammetric products products have visible have visible artefacts. artefacts. In the Incase the of case DSM, of the DSM, resulting the resulting ambiguities ambiguities in the inlocation the location of key of points key points consist consist of incorrect of incorrect altitude altitude reconstruction, reconstruction, which is which visible is north visible of north the pier of thein pierFigure in Figure 4b and4 bnear and the near central the central part of partthe pier of the itself pier in itselfFigure in 5b. Figure The5 issueb. The was issue also was presented also presented in more indetail more in detail Figure in 9. Figure 9. If key points are incorrectly located, high-density point clouds also have a large number of additional points generated outside the mean plane determined by the reconstructed area. The number of outliers increases with a decrease in the density of key points. A fragment of the extracted cloud is shown in Figure 10. The cross-section runs on the borderline of land and water. The cross-section Downloaded from mostwiedzy.pl clearly shows the change of points distribution and their density.
(a) (b)
Figure 9. DSM - Incorrectly reconstructed altitudes (a) pier area 2019, (b) pier area, and northern area in 2018.
Sensors 2020, 20, x FOR PEER REVIEW 11 of 20
not generated on the water surface. As a result, only photographs with any fixed infrastructural elements may be used for the construction of orthophotomap. Thus, around the fixed object it is only possible to generate an orthophotomap up to the maximum terrain width of the photo. The terrain size of the photo is directly proportional to the flight altitude, according to the following relationship:
ℎ푀퐴퐺퐿 퐿푊 = (2) 퐹푅 푆푊
where: ℎ푀퐴퐺퐿 —flight altitude (m), 퐹푅 —the actual focal length of the camera (mm), 퐿푊 —terrain width of the image and 푆푊—the actual width of the sensor (mm). Using this relationship, we can determine which maximum distance between the water area and fixed objects will be depicted on the orthophotomap. Figure 8a presents a fragment of the study with the aero-triangulated images and the image visible on them mapped. The described phenomenon is schematically presented in Figure 8b, where green marks the terrain area of the photo that can be included in the orthophotomap, whereas red marks the rejected photos. Fixed infrastructure elements are marked in orange.
(a) (b)
Figure 8. Generating an orthophotomap over the water area (a) during the flight in 2019, and a diagram illustrating the problem (b).
Incorrectly located key points affect the accuracy of the high-density point cloud; thus, other photogrammetric products have visible artefacts. In the case of DSM, the resulting ambiguities in the location of key points consist of incorrect altitude reconstruction, which is visible north of the pier in Sensors 2020, 20, 4000 12 of 21 Figure 4b and near the central part of the pier itself in Figure 5b. The issue was also presented in more detail in Figure 9.
Sensors 2020, 20, x FOR PEER REVIEW 12 of 20
If key points are incorrectly located, high-density point clouds also have a large number of
additional points generated outside the mean plane determined by the reconstructed area. The number of outliers increases(a) with a decrease in the density of key points.( bA) fragment of the extracted cloudFigureFigure is shown 9. 9.DSMDSM in- Incorrectly -Figure Incorrectly 10. reconstructed The reconstructed cross-section altitudes altitudes runs (a ()a pier )on pier thearea area borderline 2019, 2019, (b ()b pier) pier of area, land area, and and and northern northernwater. areaThe area cross- sectioninin 2018. 2018. clearly shows the change of points distribution and their density.
(a) (b)
FigureFigure 10. 10.Cross-section Cross-section of of a high-densitya high-density point point cloud cloud on the on shoreline,the shoreline, (a) top (a view) top withview a cross-sectionwith a cross- ofsection the extracted of the extracted cloud shown, cloud (shown,b) side view(b) side of theview extracted of the extracted cross-section. cross-section.
ThisThis phenomenon phenomenon is is especially especially pronounced pronounced on on the the borderline borderline between between the the luminous luminous surface surface of the of seathe andsea theand “dry” the “dry” land. land. Thus, Thus, if there if arethere several are several wrongly wrongly generated generated points located points randomlylocated randomly outside theoutside set of the points set reconstructingof points reconstructing a given object a given in space, object and in thespace, number and the of these number points of isthese significantly points is highersignificantly above higher water, thenabove such water, an areathen can such be an eliminated area can be using elimina a statisticalted using filter a statistical [63,64]. filter [63,64]. TheThe pointpoint cloudcloud isis filteredfiltered inin twotwo phases.phases. TheThe firstfirst phasephase consistsconsists ofof calculatingcalculating statisticalstatistical datadata ofof
thethe analysedanalysed cloud.cloud. LetLet us assume that each point point m푚i푖 describeddescribed with with coordinates coordinatesx i푥,푖,y푦i,푖,z푧i푖 belongsbelongs toto 33
Downloaded from mostwiedzy.pl spacespaceR ℝ.. Thus, Thus, a a point point cloud cloud beforebefore filtrationfiltration M푴with with a a total total number number of of points pointsM 푀p 푝cancan be be described described as follows: as follows: M = mi ; i = 1, ... , Mp; mi = (xi, yi, zi . (3) 푴 ={{푚}푖}; 𝑖 = 1, … , 푀푝; 푚푖 = (푥푖, 푦푖, 푧푖). (3)
Now, let’s mark the analysed point 푚푞 such that 푚푞 ∈ 푴 and a point in its immediate neighbourhood 푚푛such that 푚푛 ∈ 푴. Then, a set of all k points in the direct neighbourhood of the point 푚푞 can be written as 푴푛 = {푚푛1, … , 푚푛푘} on condition that each pair of points 푚푞 and 푚푛푘 meets the condition:
푝 푝 푘
√∑|mnk − 푚푞| ≤ 푑푚 (4) 1
where 푑푚 is the maximum assumed distance between the examined point and 푝 ≥ 1 (here the assumed 푝 = 2).
The average distance to all 푘 points in the neighbourhood of point 푚푞 is: 푘 1 2 푑 = ∑ √(푚 − 푚 ) (5) 푖 푘 푛푘 푞 1
and the mean value 푑푖 calculated for all points of the filtered cloud 푴 is expressed by the formula:
Sensors 2020, 20, 4000 13 of 21
Now, let’s mark the analysed point mq such that mq M and a point in its immediate neighbourhood ∈ mn such that mn M. Then, a set of all k points in the direct neighbourhood of the point mq can be ∈ written as Mn = m , ... , m on condition that each pair of points mq and m meets the condition: { n1 nk} nk tuv Xk p p m mq dm (4) nk − ≤ 1
where dm is the maximum assumed distance between the examined point and p 1 (here the assumed ≥ p = 2). The average distance to all k points in the neighbourhood of point mq is:
k r 1 X 2 d = m mq (5) i k nk − 1
and the mean value di calculated for all points of the filtered cloud M is expressed by the formula:
XMp d µ = i (6) i Mp
thus, the standard deviation of distances to all k neighbourhood points for all the point mi can be written as: s 1 XMp 2 σ = (di µ) (7) Mp i −
The first filtration step finishes with the calculation of statistical values. The next filtration stage consists in generating the resultant point cloud Mo, without outliers. Outliers are points that are located at a greater distance to k nearest neighbours than a certain threshold value T. This threshold value can be defined as follows: T = µ + ασ (8)
where α is a user-defined multiplier and can be determined experimentally for a given point cloud. The resulting point cloud after filtering can thus be written as follows:
n o Mo = mq M (µ ασ) di (µ + ασ) (9) ∈ − ≤ ≤ The point clouds obtained from photogrammetric flights by iteration have been cleaned according to a procedure adopted and described above, which means that the resulting cloud from each run has been filtered again. In this way, outliers were eliminated. The number of iterations for a given set and the coefficients for each point cloud are shown in Table7. The graphical result of the filtration described above is shown in Figure 11. Figure 11 shows the cleaned section, previously shown in Figure 10. This section clearly shows a significant reduction of outliers over the land area and a complete absence of those representing the water surface.
Table 7. Number of iterations and adopted coefficients. Downloaded from mostwiedzy.pl Campaign Coefficient 1 2 3 4 5 6 2018 k 6 6 6 6 α 1 1 1 1 2019 k 6 6 6 6 6 6 α 1 1 1 1 1 1 Sensors 2020, 20, x FOR PEER REVIEW 13 of 20