applied sciences

Article Technological Development and Application of Photo and Video Theodolites

Rinaldo Paar 1,* , Miodrag Roi´c 1 , Ante Marendi´c 1 and Stjepan Mileti´c 2

1 Department of Applied Geodesy, Faculty of Geodesy, University of Zagreb, Kaˇci´ceva26, 10000 Zagreb, Croatia; [email protected] (M.R.); [email protected] (A.M.) 2 Department of Photogrammetry, Plc., Borongajska Cesta 71, 10000 Zagreb, Croatia; [email protected] * Correspondence: [email protected]

Abstract: Theodolites are fundamental geodetic measuring instruments for all practical geodetic tasks, as well as for experimental geodetic scientific purposes. Their development has a long history. Photo and video theodolites represent the advanced development of classic theodolites. Development started in 19th century, but only in the last 15 years has commercial application been achieved in the geodetic profession. The latest development, called image-assisted total stations (IATS), is a theodolite which consists of a classic robotic total station (RTS) with integrated image sensors. It was introduced in the early 2000s. With the development of theodolites, their application became much wider; today, they can be used for structural and geo-monitoring, i.e., for the determination of static and dynamic displacements and deformations of civil engineering structures such as bridges, dams, wind turbines, and high buildings, as well as natural structures, such as mountain slopes. They can be implemented in geodetic monitoring systems, which are an integral part of engineering structural diagnosis, and they provide essential information about the current condition of the structure. This



paper describes the technological development of photo and video theodolites divided into phases
 according to the main innovations in their development. The application of modern video theodolites

Citation: Paar, R.; Roi´c,M.; (i.e., IATS) is presented through several experimental studies that were performed. The procedure of Marendi´c,A.; Mileti´c,S. conducting measurements with this kind of instrument, as well as the analysis of acquired data and Technological Development and achieved results, is elaborated. Lastly, the authors conclude, according to the achieved results, that Application of Photo and Video IATS can today be used for determination of quasi-static and dynamic displacements with small and Theodolites. Appl. Sci. 2021, 11, 3893. high amplitudes. https://doi.org/10.3390/app11093893 Keywords: photo theodolite; video theodolite; IATS; technological development; displacement and Academic Editor: Karel Pavelka deformation monitoring; structural monitoring; geo-monitoring

Received: 26 March 2021 Accepted: 22 April 2021 Published: 25 April 2021 1. Introduction

Publisher’s Note: MDPI stays neutral A theodolite is a geodetic instrument for measuring horizontal and vertical directions, with regard to jurisdictional claims in i.e., angles in the horizontal and vertical plane. There are optical and electronic theodo- published maps and institutional affil- lites [1–3]. To determine the point positions in the coordinate system, in addition to angles, iations. we must measure the distance between the instrument and the target. We can measure the distances mechanically (with chains and measuring tapes), optically (with optical distance- meters and interferometers), and electronically (with electro-optical distance-meters and electronic distance-meters (EDMs)). The accuracy of mechanically and optically measured distances is low and not appropriate for terrain characteristic point coordinate calculations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. With time, several technological developments have been incorporated into theodolites; This article is an open access article thus, the accuracy of angle and distance determination has significantly increased. Perhaps distributed under the terms and the biggest development of theodolites was its integration with an electronic distance- conditions of the Creative Commons meter (EDM), which emerged around 1940 and became commercially available in the late Attribution (CC BY) license (https:// 1960s [4]. The type of EDM incorporated into modern total station instruments is commonly creativecommons.org/licenses/by/ of the electro-optical type, using infrared and laser light as a carrier signal [3]. Instruments 4.0/). which can measure angles and distances simultaneously, as well as record the results and,

Appl. Sci. 2021, 11, 3893. https://doi.org/10.3390/app11093893 https://www.mdpi.com/journal/applsci Appl. Sci. 2021, 11, x FOR PEER REVIEW 2 of 29

as a carrier signal [3]. Instruments which can measure angles and distances simultane- ously, as well as record the results and, to a certain extent, process them, are called elec- Appl. Sci. 11 2021,tronic, 3893 tachymeters or total stations (TS) [2]. TS instruments are electronic digital theodo- 2 of 29 lites integrated with EDM instruments and electronic data collectors, which are aimed at replacing manual field data recording; they are capable of providing electronic angle read- ings, as well as distanceto a certain measurements extent, process [3]. them, are called electronic tachymeters or total stations (TS) [2]. TS instruments are electronic digital theodolites integrated with EDM instruments and Today, many different sensors and measurement methods are combined in TS, such electronic data collectors, which are aimed at replacing manual field data recording; they as highly accurateare angle capable reading, of providing electronic electronic distance angle readings,measurements as well (EDM) as distance to reflectors measurements [3]. and to any other surfaceToday, (reflectorless many different EDM), sensors tilt correction and measurement by two-axis methods inclinometers, are combined dif- in TS, such ferent types of motorizationas highly accurate to drive angle both reading, the horizontal electronic distance and the measurements vertical motion (EDM) of the to reflectors instruments, servo,and piezo, to any and other magnetic surface (reflectorlessmotors (robotic EDM), total tilt station—RTS), correction by two-axis image CCD inclinometers, (charge-coupled differentdevice) typesor CMOS of motorization (complemen to drivetary metal–oxide–semiconductor) both the horizontal and the vertical sensors motion of the (CCD or CMOS) instruments,for autofocus, servo, automated piezo, and aimi magneticng (i.e., motors automatic (robotic target total recognition station—RTS), (ATR) image CCD and tracking of signal(charge-coupled points), integration device) or CMOS with global (complementary navigation metal–oxide–semiconductor) satellite system (GNSS) sensors positioning, wireless(CCD communication or CMOS) for autofocus, and operation automated using aiming a controller, (i.e., automatic additional target cameras recognition (ATR) and tracking of signal points), integration with global navigation satellite system (GNSS) for documentation (image-assisted total station—IATS), and IATS with a scanning func- positioning, wireless communication and operation using a controller, additional cameras tion (image-assistedfor documentation scanning total (image-assisted station—IASTS). total station—IATS),Due to the rapid and hardware IATS with adevelop- scanning function ment, as can be (image-assistedseen in Figure scanning 1, these totaldiffer station—IASTS).ent sensor classes, Due toeach the rapidwith hardwaretheir specific development, advantages, can asbe canunified, be seen utilized, in Figure and1, these are differentfused as sensor one single classes, (nearly) each with universal their specific instru- advantages, ment [5]. Moderncan TS be are unified, multi-sensor utilized, systems and are fused which as can one determine single (nearly) the universal three-dimensional instrument [5]. Mod- coordinates of targetern TS points are multi-sensor by combining systems horizontal which can angle, determine vertical the three-dimensional angle, and distance coordinates of measurements [6].target points by combining horizontal angle, vertical angle, and distance measurements [6]. Figure 1 shows theFigure technological1 shows the development technological of development TS over the ofyears, TS over i.e., thefrom years, the mid- i.e., from the mid-20th century until today. The main hardware and software, sensor integration, and 20th century until today. The main hardware and software, sensor integration, and their their impact on instrument performances and operator efficiency for conducting the mea- impact on instrumentsurements performances in the field and are shown.operator It efficiency is evident for that, conducting following thethe integration measure- of EDM ments in the fieldwith are theshown. basic It theodolite is evident in that, the following 1960s upon the discovering integration tachymeters, of EDM with the the dimensions basic theodolite inand the weight 1960s ofupon the instrumentsdiscovering decreased, tachymeters, as well the asdimensions the expert and efficiency weight regarding of the the instruments decreased,measuring techniquesas well as andthe ex timepert for efficiency conducting regarding measurements the measuring in the field. tech- Contrarily, niques and time instrumentfor conducting performance measurements and the possibilitiesin the field. for Contrarily, surveying instrument and monitoring perfor- projects have mance and the possibilitiesrapidly increased. for surveying and monitoring projects have rapidly increased.

Figure 1. Total stationFigure technological 1. Total station development technological advances. development advances.

Different types Differentof motorization types of developed motorization to developed drive instruments to drive instruments in horizontal in horizontal and and vertical motion havevertical been motion specially have beendesign speciallyed and designed implemented and implemented in RTS to automatically in RTS to automatically search for reflectors and, later, for reflector tracking. Most commonly, RTS are used for search for reflectors and, later, for reflector tracking. Most commonly, RTS are used for highly precise measurements such as displacements and deformation monitoring of civil highly precise measurementsengineering structures such as anddisplacements the Earth’s surface. and deformation monitoring of civil engineering structures and the Earth’s surface. Appl. Sci. 2021, 11, 3893 3 of 29

RTS with additional integrated cameras in the telescope are commonly denoted as image-assisted total station (IATS) [7]. In the literature, a number of alternative terms exist for this class of instruments [8]: photo theodolite, video theodolite/tacheometer, image- assisted photogrammetric scanning station, and others. Differentiation is often made between a video and an image or photo. This fact was valid in the past, considering analog imaging devices introduced at the end of 19th century. Static instruments on a leveled, rotatable platform with the main function of taking images using a photogrammetric camera, supported by a theodolite for determining the elements of the exterior orientation, are called photo theodolites [9,10]. Currently, IATS have possible applications in semiautomated object reconstruction sys- tems [11], fully automated deformation monitoring systems [12], industrial measurement systems [13], measurements of vibration amplitudes by means of high-frequency image measurements [14], capture of additional information such as high-frequency motions or intensity fluctuations of patterns using image sensor to derive the temperature gradient of the atmosphere as a decisive influence parameter for angular refraction effects [15] and monitoring of cracks [14,16]. This study reviews and describes the technological development of photo and video theodolites divided into phases according to the main innovations in their development (Section2). Emphasis is placed on commercially developed instruments; however, proto- types from academia are also analyzed, since they often inspire technological developments. The basic principles of the modern IATS, camera and system calibration, and measurement procedures and data processing approaches are explained (Section3). The application of IATS, as the successor to photo and video theodolites, is presented through experimental and in situ studies previously performed (Section4). Lastly, a discussion with the main conclusions from conducted studies is presented.

2. Technological Development We can differentiate two main phases in the development of photo and video theodo- lites. The first phase is considered when the theodolite was a supporting tool for pho- togrammetric cameras, called photo theodolites, and the second phase is considered when the camera became a supporting tool for theodolites, i.e., tachymeters and total stations, called video theodolites. The second phase involving the technological development of video theodolites can be divided into four main phases according to the main innovations in their development. In this section, each phase is described, and the main innovations are explained.

2.1. Development until 1940 Photo theodolites have been known for more than 150 years in photogrammetry. The main credit for the introduction of photogrammetry goes to Aime Laussedat, an officer in the Engineer Corps of the French Army. He is known as the “Father of Photogrammetry” as he constructed the first usable cameras for terrestrial work in 1851 [17]. He generated the first maps by means of photographs which he took from a balloon. His remaining research focused on terrestrial photogrammetry, designing the first photo theodolite which consisted of a camera and theodolite [18]. The end of 19th century was significant for the development of terrestrial photogrammetry, which was applied in many mapping projects. In 1889, the Italian army developed the photo theodolite (Figure2, left). Sebastian Finsterwalder, in 1899, wrote his dissertation on the “fundamental geometry of photogrammetry”, in which he solved the problem of determining the position of two camera stations, i.e., stereopairs, independently of terrain measurements from four points identified on both photographs, as well as the methods of relative and absolute orientation. He also developed his own photo theodolite (Figure2, middle). German Carl Pulfrich, with the construction of his photo theodolite (Figure2, right), designed the first modern stereo comparator which used the principles of stereophotogrammetry and x–y coordinate scales [17,18]. At first, a Appl. Sci. 2021, 11, 3893 4 of 29 Appl.Appl. Sci. Sci. 2021 2021, 11, 11, x, xFOR FOR PEER PEER REVIEW REVIEW 4 4of of 29 29

photogrammetric camera was combined with components such as an aiming device, pitch [17,18].[17,18]. At At first, first, a aphotogrammetric photogrammetric camera camera was was combined combined with with components components such such as as an an circle, level, and tripod [19]. aimingaiming device, device, pitch pitch circle circle, level,, level, and and tripod tripod [19]. [19].

FigureFigureFigure 2. 2.2. Photo Photo theodolite theodolite from from Italian Italian Italian army army army (left ( (leftleft), ),Finsterwalder), Finsterwalder Finsterwalder photo photo photo theodolite theodolite theodolite (middle (middle (middle), ),and ),and and Pulfrich photo theodolite (right) [18]. PulfrichPulfrich photo photo theodolite theodolite (right (right) )[[18].18].

TheTheThe Wild WildWild company companycompany in inin 1922 19221922 constructed constructedconstructed the thethe first firstfirst commercial commercialcommercial photo photophoto theodolite, theodolite,theodolite, Wild WildWild P30P30P30 (Figure (Figure(Figure 3,3 3,, left) left) left) [20]. [ [20].20]. The The instrument instrument was waswas a acombination combination of ofof a aacamera camera and and theodolite theodolite and, and,and, asasas such, such,such, was waswas unique uniqueunique in inin the thethe world. world.world. Wild’s Wild’sWild’s ph phototheodolitephototheodoliteototheodolite was waswas the thethe most mostmost accurate accurateaccurate over overover the thethe nextnextnext three threethree decades. decades.decades. The TheThe magnification magnificationmagnification of ofof the the the telescope telescope telescope was was was 28×, 28 28×,× ,and andand the thethe direction directiondirection reading readingreading withwithwith an anan optical opticaloptical micrometer micrometermicrometer was waswas 1”. 1”.1”. The TheThe focal focalfocal length lengthlength of ofof the thethe camera cameracamera lens lenslens was waswas 165165 165 mm.mm. mm. The TheThe cameracameracamera and andand the thethe theodolite theodolitetheodolite weighed weighedweighed 27.5 27.527.5 kg, kg,kg, while whilewhile the thethe weight weightweight of ofof the thethe complete completecomplete measuring measuringmeasuring equipmentequipmentequipment with with all all the the necessary necessary aids aids aids was was was about about about 65 65 65kg. kg. kg. The The The instrument instrument instrument was was was applicable applicable applicable to to alltoall large all large large and and and small small small geodetic geodetic geodetic tasks. tasks. tasks. The The The instrument’s instrument’s instrument’s reliability reliability reliability during during during field field field work work in inin the thethe mostmostmost difficult difficultdifficult conditions, conditions,conditions, which whichwhich are areare common commoncommon during duringduring surveys surveyssurveys in inin high highhigh mountains, mountains,mountains, was waswas one oneone ofofof its itsits main mainmain advantages. advantages.advantages. Three Three Three years years later, later, WildWild Wild introducedintroduced introduced a a new anew new model model model of of photoof photo photo theodolite theodo- theodo- litecalledlite called called the the Wildthe Wild Wild FT9 FT9 (FigureFT9 (Figur (Figur3, right).e e3, 3, right). right). The The mechanical The mechanical mechanical construction construction construction of the of of theodolitethe the theodolite theodolite and andtheand the camera the camera camera simplified simplified simplified the determinationthe the determination determination of the of of camera’sthe the camera’s camera’s orientation orientation orientation for photogrammetric for for photogram- photogram- metricpostprocessing.metric postprocessing. postprocessing.

Figure 3. Wild photo theodolites P30 (left) and FT9 (right) [20]. FigureFigure 3. 3. WildWild photo photo theodolites theodolites P30 P30 (left (left) )and and FT9 FT9 (right (right))[ [20].20].

2.2.2.2.2.2. Development DevelopmentDevelopment from fromfrom 1941–1980 1941–19801941–1980 AfterAfterAfter World WorldWorld War War War II, II, a a theodolite theodolite namednamed named ElectronicElectr Electroniconic Eye Eye Eye (Figure (Figure (Figure4, 4, left) 4, left) left) was was was developed developed developed at attheat the the Institute Institute Institute of of Applied of Applied Applied Geodesy Geodesy Geodesy in Frankfurtin in Fran Frankfurtkfurt for automatedfor for automated automated aiming aiming aiming based based based on the on on difference the the dif- dif- Appl. Sci. 2021, 11, 3893 5 of 29 Appl. Sci. 2021, 11, x FOR PEER REVIEW 5 of 29

ofference twocurrents of two currents in a photoelectric in a photoelectric cell.This cell. methodThis method was was further further developed developed to enableto ena- bearingble bearing accuracies accuracies of 1” of as1” aas mean a mean of sixof six sets sets of of directions directions in in the the late late 1950s 1950s [ 21[21].]. TestingTesting gavegave veryvery goodgood results;results; thus,thus, thethe companycompany AskaniaAskania producedproduced aa modifiedmodified theodolitetheodolite basedbased onon ElectronicElectronic Eye—theEye—the AskaniaAskania TrackingTracking System System (Figure (Figure4 ,4, right). right).

FigureFigure 4.4. Electronic Eye Eye from from the the Frankfur Frankfurtt Institute Institute of of Applied Applied Geodesy Geodesy (left (left) and) and the the Askania Askania TrackingTracking System System ( right(right)[) 21[21,22].,22].

WhileWhile ElectronicElectronic EyeEye couldcould onlyonly measuremeasure staticstatic targets,targets, thethe AskaniaAskania TrackingTrackingSystem System alreadyalready hadhad thethe abilityability toto registerregister thethe orbitorbit ofof movingmoving objectsobjects (aircraft(aircraft oror missile)missile) duringduring WW2.WW2. TheThe AskaniaAskania TrackingTracking SystemSystem waswas usedused asas aa trackingtracking systemsystem fromfrom thethe latelate 1950s1950s toto 19731973 [[23].23]. Later,Later, thethe typetype ofof theodolitetheodolite shownshown onon thethe leftleft ofof FigureFigure beganbegan toto bebe producedproduced inin SwitzerlandSwitzerland under under the the name name Contraves Contraves EOTS EOTS (electro-optical (electro-optical tracking tracking system) system) and and was waslater later produced produced in China in China [22]. [The22]. instrument, The instrument, called called a cinetheodolite, a cinetheodolite, was used was to used record to recordimportant important data during data during testing testing of missiles of missiles and and aircrafts, aircrafts, as as well well as as other experimentalexperimental weaponweapon systems.systems. Ballistic Ballistic cameras cameras that that were were used used in in satellite satellite geodesy geodesy very very often often had had in- integratedtegrated CCD CCD image image sensors sensors for for video video recording, recording, and and they they can canalso also be structurally be structurally con- consideredsidered as video as video theodolites theodolites [22]. [22 ]. WithWith thethe aimaim ofof recordingrecording thethe landinglanding ofof aircraftaircraft usingusing thethe instrument,instrument, inin 1972,1972, aa portableportable MinilirMinilir systemsystem waswas developeddeveloped in in Paris Paris [ 22[22].]. TheThe systemsystem waswas developeddeveloped as as partpart of of thethe internalinternal research research project project SAT SAT (Soci (Sociétéété Anonyme Anonyme de Tdeél éTélécommunications),communications), and and it was it was not onlynot only a geodetic a geodetic advancement advancement but also but aalso military a military tool for tool automatic for automatic monitoring monitoring of infrared of in- radiation,frared radiation, such as such aircraft as andaircraft missile and exhausts.missile exhausts. The aiming The was aiming based was on based the detection on the de- of aircrafttection headlightsof aircraft headlights using a very using sensitive a very infrared sensitive sensor infrared that sensor recorded that the recorded frequency the and fre- phase-modulatedquency and phase-modulated signal. This instrumentsignal. This has inst beenrument used has since been the used 1980s since in combination the 1980s in withcombination various distance-meterswith various distance-meters such as the IBEO such Fennel as the electronic IBEO Fennel distance-meter electronic (EDM) distance- or themeter AGA (EDM) Geodimeter-112 or the AGA EDM.Geodimeter-112 Depending EDM. on which Depending EDM wason which in place, EDM the was instrument in place, wasthe instrument referred to aswas AGA-Minilir referred to (Figureas AGA-Minilir5, middle) (Figure or Fennel-Minilir. 5, middle) or This Fennel-Minilir. instrument was This usedinstrument as early was as 1980 used in as the early Dutch as 1980 delta in project the Dutch for the delta coastal project navigation for the coastal of transport navigation ships. Inof combinationtransport ships. with In a filmcombination camera, thewith system a film became camera, known the system as a cinetheodolite, became known which as a wascinetheodolite, used on launch which sites was Kourou used on (Guiana launch Space sites Centre)Kourou and(Guiana Cape Space Canaveral. Centre) In and civil Cape use, MinilirCanaveral. has beenIn civil used use, for Minilir the calibration has been of used instrument for the landingcalibration systems of instrument [24]. landing systemsMinilir [24]. was followed, in 1983, by the Krupp Atlas-Elektronik instrument known as Polarfix (Figure5, right). The laser measuring principle for short-range high-accuracy Minilir was followed, in 1983, by the Krupp Atlas-Elektronik instrument known as dynamic survey applications was the basis of Polarfix. Polarfix had several advantages and Polarfix (Figure 5, right). The laser measuring principle for short-range high-accuracy dy- improvements compared to the Minilir system. While Minilir required a heat source to find namic survey applications was the basis of Polarfix. Polarfix had several advantages and the target, Polarfix only needed a passive reflective prism to achieve target tracking. The improvements compared to the Minilir system. While Minilir required a heat source to vertical circuit was automatically compensated for, and the instrument could be operated find the target, Polarfix only needed a passive reflective prism to achieve target tracking. on a 12 V source instead of a 220 V source, which allowed the use of batteries. Its weight and The vertical circuit was automatically compensated for, and the instrument could be op- volume were twofold lower compared to Minilir. The system provided polar coordinates erated on a 12 V source instead of a 220 V source, which allowed the use of batteries. Its Appl. Sci. 2021, 11, x FOR PEER REVIEW 6 of 29

Appl. Sci. 2021, 11, 3893 6 of 29

weight and volume were twofold lower compared to Minilir. The system provided polar coordinates for position fixing, incorporated a fully automatic operator-free tracking func- tion,for position and ensured fixing, range/azimuth incorporated fixing a fully to automatic 10 cm + 5 cm/km operator-free accuracy tracking from a function, single shore- and stationensured linked range/azimuth to shipborne fixing telemetry to 10 cm and + 5beam cm/km reflector accuracy facilities. from aThe single operating shore-station range dependedlinked to shipborne on weather telemetry and other and factors, beam reflectorbut its typical facilities. basic The range operating was just range under depended 3 km, whichon weather could and be extended other factors, to 5 but km its or typical more. basicThis instrument range was just was under primarily 3 km, designed which could for trackingbe extended ships to on 5 kminland or more. waterways This instrument [25]. The 1970s was primarily and 1980s designed were marked for tracking by the shipsinte- on inland waterways [25]. The 1970s and 1980s were marked by the integration of EDM gration of EDM and electronics in general into theodolites, which also influenced the in- and electronics in general into theodolites, which also influenced the introduction of a new troduction of a new name for such instruments, i.e., electronic theodolites or electronic name for such instruments, i.e., electronic theodolites or electronic tachymeters (ETs) [26]. tachymeters (ETs) [26].

Figure 5. Contraves EOTS cinetheodolite ( (leftleft))[ [23],23], AGA-Minilir AGA-Minilir cinetheodolite cinetheodolite ( (middlemiddle))[ [24],24], and and Krupp Krupp Atlas Atlas Polarfix Polarfix (right))[ [22].22].

2.3.2.3. Development Development from from 1981–1990 1981–1990 DevelopmentDevelopment during during the the 1980s 1980s placement placement emphasis emphasis on on theodolite theodolite measurement measurement sys- sys- temstems (TMSs). (TMSs). ETs ETs combined combined with with integrated integrated ca cameras,meras, i.e., i.e., image image CCD CCD sensors, sensors, were were used toto measure measure spatial spatial forward forward intersections usin usingg two or more instruments. The automatic aimingaiming at at fixed fixed target target points or or projected laser spots was realized using image analysis techniquestechniques [8]. [8]. Most Most of of these these devices devices we werere used used for for industrial industrial applications. applications. InIn 1981, 1981, the the Geodetic Geodetic Institute Institute of of the the Univer Universitysity of of Hannover Hannover took up the the idea idea of of using using four-quadrantfour-quadrant diodes diodes as as part part of of the the Collaborative Collaborative Research Research Center Center 149 149 within within a a subproject subproject calledcalled “automatic “automatic determination determination of of the the position position of an an object object at sea sea using using a tachymeter”. A A semiconductorsemiconductor position-sensitive position-sensitive sensor sensor buil builtt into into the the image image plane plane of of a tachymeter was usedused to to detect detect the the center center of of the the light light source source located located on on a monitored a monitored ship. ship. With With the the help help of built-inof built-in servomotors, servomotors, the the instrument instrument could could monitor monitor the the ship, ship, always always keeping keeping the the light sourcesource in in the the middle middle of of the the field field of of view view (F (FOV).OV). This This development development was was based based on on the the AGA AGA GeodimeterGeodimeter 700, 700, which which had had infinite infinite screws screws for for fine fine adjustment, adjustment, named named GEOROBOT, GEOROBOT, and GEOROBOTGEOROBOT II II was was later later developed developed on on the the basis basis of of the the Geodimeter Geodimeter 710 710 instrument instrument [27,28]. [27,28]. TwoTwo instruments which which marked marked this this period period were were presented presented at at almost almost the the same same time, time, i.e.,i.e., Kern Kern E2-SE E2-SE (Figure (Figure 66,, left) introduced inin 1985 [[29]29] and and Wild Wild TM-3000V TM-3000V (Figure (Figure 66,, right) introducedintroduced shortly shortly after after [30]. [30]. These These instrume instrumentsnts were were developed developed through through projects from KernKern and and Wild Wild and and were were actually actually called called Kern Kern SPACE SPACE and and Wild Wild ATMS during the project phase.phase. The growing demandsdemands of of manufacturers manufacturer ins in the the automotive automotive and and aerospace aerospace industries, indus- tries,who usedwho used theodolite theodolite measurement measurement systems syst duringems during assembly assembly and qualityand quality control, control, were werebased based on measuring on measuring spatial spatial forward forward intersections intersections using using two two or more or more instruments. instruments. To Tosatisfy satisfy these these demands, demands, Kern Kern and and Wild Wild integrated integrated CCD CCD image imagesensors sensors andand digital image acquisitionacquisition into into servomotorized servomotorized theodolites. theodolites. These These so-called so-called video video theodolites theodolites had had pan- pan- focalfocal telescopes, telescopes, which which means means that that the the FOV FOV changed changed as as a a function function of of distance distance to to the the target target × point.point. Kern Kern E2-SE E2-SE had had a a CCD CCD image image sensor wi withth a resolution of 576 × 684684 px, px, and the image was recorded in the instrument’s internal memory. It was the first video theodolite Appl. Sci. 2021, 11, x FOR PEER REVIEW 7 of 29 Appl. Sci. 2021, 11, 3893 7 of 29

image was recorded in the instrument’s internal memory. It was the first video theodolite inin thethe worldworld withwith thethe possibilitypossibility of of automatically automatically aiming aiming at at a a signaled signaled target target [ 2[2].]. The The CCD CCD imageimage sensor sensor was was used used for for autofocusing autofocusing and and for fo determiningr determining the the target target position position relative relative to theto the optical optical axis axis of theof the telescope. telescope. The The instrument instrument had had three three integrated integrated servomotors, servomotors, two two forfor horizontalhorizontal and vertical axes axes and and one one for for fo focusing,cusing, i.e., i.e., autofocus. autofocus. The The motors motors were were de- ± designedsigned to to enable enable a apositioning positioning accuracy accuracy of of up up to ±0.1”0.1” to to be be achieved, at speedsspeeds ofof upup toto 54◦ per second [31]. Once the autofocus was accomplished and the geometrical relationship 54° per second [31]. Once the autofocus was accomplished and the geometrical relation- between the target point and the instrument optical axis was determined using image ship between the target point and the instrument optical axis was determined using image processing algorithms, the instruments automatically drove the theodolite toward the processing algorithms, the instruments automatically drove the theodolite toward the tar- target point and the measurements could be performed. A signaled target point realized get point and the measurements could be performed. A signaled target point realized by by a projected laser point or a fixed marking was automatically aimed at by two or more a projected laser point or a fixed marking was automatically aimed at by two or more corresponding instruments using image evaluation, allowing its 3D spatial coordinates to corresponding instruments using image evaluation, allowing its 3D spatial coordinates to be determined [22]. be determined [22].

FigureFigure 6.6.Video Video theodolitestheodolites KernKern E2-SEE2-SE ( left(left)[) 29[29]] and and Wild Wild TM3000V TM3000V ( right(right)[) 30[30].].

ShortlyShortly after after Kern, Kern, the the Wild Wild company company introduced introduced its video its theodolite video theodolite model TM3000V. model TheTM3000V. motorized The videomotorized theodolite video TM3000Vtheodolite wasTM3000V a further was development a further development of the precision of the theodoliteprecision theodolite T3000. In T3000. addition In toaddition the motors to the for motors the axes for andthe axes the focusingand the focusing lens, special lens, elementsspecial elements of the motorized of the motorized video theodolite video weretheodolite the camera were andthe thecamera processor and the integrated processor in theintegrated theodolite. in the Wild theodolite. TM3000V Wild had TM3000V a CCD image had sensora CCD withimage a resolutionsensor with of a500 resolution× 582 px of. The500 coaxial× 582 px. configuration The coaxial of configuration a camera (integrated of a camera into the(integrated telescope) into using the theodolitetelescope) optics using wastheodolite better, optics but also was technically better, but more also techni complicatedcally more than complicated biaxial implementation than biaxial implemen- onto the telescopetation onto leading the telescope to parallax, leading which to parallax, is particularly which harmfulis particularly in situations harmful where in situations range- finderswhere range-finders operate at close operate target at distances. close target When distances. using the When coaxial using configuration, the coaxial configura- complex beamtion, complex splitter elementsbeam splitter must elements be installed. must be The installed. Leica TM3000V The Leica video TM3000V theodolite video theod- is the best-performingolite is the best-performing example, widely example, used widely in academic used in researchacademic [11 research]. The image [11]. The of theimage FOV of ofthe the FOV telescope of the ontelescope the CCD on arraythe CCD contains array the cont imageains the of theimage targeted of the point, targeted as well point, as as a referencewell as a framereference built frame into thebuilt telescope into the totelescop replacee theto replace reticle. the This reticle. frame This defines frame the defines target linethe target of the line telescope of the telescope with the centerwith the of center the lens of the system, lens system, and it is and used it is at used the sameat the timesame totime adjust to adjust the telescope the telescope and camera and camera axis. Usingaxis. Using the CCD the CCD camera, camera, a wide-angle a wide-angle lens canlens ◦ ◦ initiallycan initially be used be used to display to display an overview an overview image im ofage the of object the object space space with awith size a of size 9 ×of 129° ×. These12°. These wide-angle wide-angle optics optics can becan exchanged be exchanged for the for opticsthe optics of a of precision a precision telescope telescope via anvia opticsan optics coupler. coupler. Instead Instead of of the the wide-angle wide-angle FOV, FOV, however, however, a a narrowly narrowly limited limited measuring measuring FOVFOV isis available.available. SwitchingSwitching betweenbetween thethe measuringmeasuring andand wide-anglewide-angle FOVFOV isis controlledcontrolled byby thethe software.software. SwitchingSwitching toto thethe measuringmeasuring FOVFOV enablesenables highhigh increasesincreases inin accuracy,accuracy, sincesince detailsdetails ofof objectsobjects thatthat areare farfar awayaway are are now now displayed displayed on on the the entire entire sensor sensor surface. surface. These instruments were only used for special applications and not for standard sur- veys due to their complex morphology, which required an image display unit and a con- trol unit in addition to the instrument [32]. With the launch of laser tracker systems in the Appl. Sci. 2021, 11, 3893 8 of 29

Appl. Sci. 2021, 11, x FOR PEER REVIEW 8 of 29

These instruments were only used for special applications and not for standard surveys due to their complex morphology, which required an image display unit and a 1990s,control video unit intheodolites addition towere the replaced instrument within [32]. a With few theyears; launch this ofremained laser tracker a trend systems until 2005,in the when 1990s, Topcon video theodolitesintroduced were the first replaced commercially within a fewknown years; IA thisTS model remained GPT-7000i, a trend whichuntil 2005, was the when forerunner Topcon introduced of today’s state-of-t the first commerciallyhe-art IATS. However, known IATS technical model innovations GPT-7000i, thatwhich were was necessary the forerunner for automation of today’s using state-of-the-art video images IATS. and However, were, therefore, technical innovationsintroduced onthat the were market necessary have forbeen automation preserved using to this video day, images for example, and were, the therefore,motorized introduced endless fine on drivesthe market for instrument have been preservedcontrol, as towell this as day, reflector for example, detection the and motorized tracking endless using fineintegrated drives imagefor instrument sensors, control,which asare well common as reflector today, detection from the and second tracking generation using integrated of the imageWild TM3000V.sensors, which As in are the common TOPOMAT today, project, from thea target second illuminator generation in of the the form Wild of TM3000V. an infrared As collimatorin the TOPOMAT was installed project, here a targetfor permanent illuminator deformation in the form measurement of an infrared on collimatorreflectors. This was diodeinstalled integrated here for permanentin the telescope deformation illuminat measuremented the roughly on reflectors. approached This reflector. diode integrated The re- flectedin the telescopesignal was illuminated captured on the the roughly infrared approached sensitive CCD reflector. array Theand reflectedused for precise signal wastar- getingcaptured [30]. on the infrared sensitive CCD array and used for precise targeting [30]. TheThe concept involving involving the the difference difference of tw twoo currents in a photoelectric cell, named “Electronic Eye”, become commercially availableavailable in 1987 when Geodimeter presented the modelmodel 4000 4000 “One-Man-System” “One-Man-System” with automated target aiming, being the first first instrument thatthat required required only only one expert to operate it (Figure7 7))[ [22].22]. Since then, automated target aimingaiming and tracking havehave becomebecome standard standard functions functions in in all all TS, TS, i.e., i.e., RTS. RTS. Integrated Integrated cameras cam- erasand and photogrammetric photogrammetric methods methods became became supporting supporting tools tools for for theodolites. theodolites. ThisThis system worked using a theodolite-like active target point receiver, whereby the telescope of this worked using a theodolite-like active target point receiver, whereby the telescope of this unit, called the remote processing unit (RPU), had to be aligned with the total station, unit, called the remote processing unit (RPU), had to be aligned with the total station, called the control and processing unit (CPU), in order to facilitate a coarse search. called the control and processing unit (CPU), in order to facilitate a coarse search.

Figure 7. Geodimeter model 4000, i.e., the first “One-Man-System” in the world that required only one expert to operate Figure 7. Geodimeter model 4000, i.e., the first “One-Man-System” in the world that required only one expert to operate it [22]. it [22].

2.4.2.4. Development Development from from 1991–2000 1991–2000 AvailableAvailable prototypesprototypes ofof videovideo theodolites theodolites (Figure (Figure6) 6) in thein the 1990s 1990s encouraged encouraged intensive inten- siveresearch research on their on their application application with with high hi degreesgh degrees of automation. of automation. Equipped Equipped with with motors, mo- tors,video video theodolites theodolites made made it possible it possible to increase to increase the dynamism the dynamism of computer-controlled of computer-controlled mea- measurements,surements, turning turning a video a video theodolite theodolite into ainto robot a [robot33]. Built-in [33]. Built-in sensors sensors provided provided a wealth a wealthof data of that data demanded that demanded to be processed to be processed meaningfully meaningfully to obtain to obtain the necessary the necessary information. infor- mation.Electronic Electronic reading ofreading directions of directions and lengths and was lengths around was for around a long time,for a long but the time, processing but the processingof data obtained of data using obtained built-in using cameras built-in was cameras still inwas its still initial in its stages. initial The stages. application The appli- of cationdigital of image digital processing image processing methods andmethods computer and computer vision was vision investigated was investigated by the academic by the academiccommunity. community. Initial research Initial focused research on focused detecting on artificial detecting targets artificial and accuratetargets and aiming accurate with aimingthe help with of cameras the help [ 34of]. cameras [34]. Aiming at and measuring unsignalized targets proved to be a special challenge [35]. To achieve this, solutions based on artificial intelligence were developed [36]. Until then, Appl. Sci. 2021, 11, 3893 9 of 29

Aiming at and measuring unsignalized targets proved to be a special challenge [35]. To achieve this, solutions based on artificial intelligence were developed [36]. Until then, the expert/operator was in charge of sighting and deciding on the characteristic points to be measured. By implementing this knowledge into software solutions, artificial intelligence was applied, and expert systems were developed. Initially, development was limited to smaller and simpler processes that occurred in theodolite measurements [37]. Special attention was paid to the necessary sharpening of the image by developing autofocus algorithms [35,37]. The developed processes and procedures were later implemented in commercial solutions of instrument manufacturers. This freed observers from repetitive operations, whereby they could now focus on more complex tasks. By automating the measurements, the level of achieved accuracy was increased. Computer-aided measurements and data processing provided greater accuracy than could be achieved visually by the observer. The developed systems could be applied to measurements in the industry. It was no longer necessary to bring the object to the measurement place since the system could be transported. External control of the developed systems allowed measuring objects at hazardous places. Since unsignalized targets could be targeted, measuring of inaccessible objects also became possible. At the very beginning of this epoch, in 1991, the automatic measuring system TOPO- MAT was developed in Zurich. In collaboration with Wild, it was upgraded with a CCD image sensor to detect the target [22]. Its main features were that it could work independently, automatically, or via a controller. Other functions were also available via the controller, such as activation, deactivation, and change of measurement method [38]. GEOROBOT and TOPOMAT had encouraged the development of commercial tracking TS. The transition from the 1980s to the 1990s was also marked by the development of the ATR function, which enabled the further development of IATS instruments. In 1991, Topcon presented its first motorized, auto-tracking total station AP-S1. In 1993, Topcon released its successor called model AP-L1, and 1996 saw the emergence of the new model AP-L1A, the so-called “one-person surveying system” [39]. This RTS differed from the instruments and developments presented before in that the fine aiming, similar to the structure of a television image, worked with line-deflecting optical elements (AO device) and could, thus, evaluate an image of the reflector illuminated by the instrument [40]. This system worked in conjunction with radio communication, a handheld data collector, and TDS Survey Pro Software. It produced a unique surveying system that enabled all field surveying tasks to be carried out by one expert working on their own. The Wild company, i.e., Leica, from the beginning of the 1990s presented its first motorized self-aiming surveying system with an automatic target recognition function that worked without an external evaluation unit in 1996, i.e., the model TCA1800. This system also worked with an imaging CCD sensor and evaluated the illuminated image of the reflector using a best-fit algorithm. With this instrument, a new type of compact 360◦ all-round reflector was developed [22].

2.5. Development from 2001 until Today The forerunner of today’s state-of-the-art commercial IATS systems was Topcon’s model GPT-7000i from 2005 (Figure8). Topcon integrated the camera into a TS, which created a new generation of video theodolites, named IATS. The instrument was equipped with two different VGA cameras. One camera was wide-angle with an FOV of 28◦ × 22◦ and fixed focus, located on the telescope, allowing rotation to be synchronized and the field working area to be displayed. The second camera was coaxial, and it recorded the details of the terrain seen through the magnification of the telescope; thus, it included variable focus. It had an FOV that matched the telescope FOV (1◦300 × 1◦300)[41]. In 2007, Topcon introduced the successor of the first series model GPT-9000Ai. The greatest technological development compared to the GPT-7000i was manifested as the possibility of automatic focusing of signal points and remote guidance by the controller, which eliminated the need Appl. Sci. 2021, 11, x FOR PEER REVIEW 10 of 29 Appl. Sci. 2021, 11, 3893 10 of 29

first RTS that enabled the taking of photographs and the possibility of scanning as a for an observer, making it a so-called “one-man station”. It was also the first RTS that unique function within the IATS system. The instrument could scan up 20 points per sec- enabled the taking of photographs and the possibility of scanning as a unique function ond [42]. These types of IATS have been produced by Topcon for three generations and within the IATS system. The instrument could scan up 20 points per second [42]. These are known as Imaging Stations, with the latest model called IS3. Both cameras had an types of IATS have been produced by Topcon for three generations and are known as Imagingimage sensor Stations, with with a resolution the latest modelof 1280 called × 1024 IS3. pixels, Both and cameras they hadcould an also image record sensor video with at a10 resolution frames per of 1280second× 1024 (fps), pixels, which and could they be could transmitted also record to the video station at 10 control frames screen per second or to (fps),the controller which could screen. be transmitted to the station control screen or to the controller screen. TrimbleTrimble introduced introduced itsits firstfirst IATS IATS in in 2007, 2007, the the model model VX VX Spatial Spatial Station Station (Figure (Figure8 ).8). It It waswas basicallybasically anan S6S6 modelmodel withwith addedadded functionalityfunctionality ofof 3D3D scanningscanning andand digitaldigital imaging.imaging. VXVX delivereddelivered imagesimages thatthat were were accurately accurately georeferenced. georeferenced. TheThe instrument instrument was was equipped equipped withwith a a 3 3 MPx MPx color color camera, camera, with with fixed fixed focus focus and and an an FOV FOV of of 16.5 16.5°◦ × ×12.3 12.3°.◦. TheThe cameracamera was was locatedlocated onon thethe telescope.telescope. ThisThis cameracamera waswas laterlater implementedimplemented inin allall total total station station models models thatthat TrimbleTrimble released, released, including including today’s today’s Trimble Trimble S9 S9 series. series. The The features features of of postprocessing postprocessing photogrammetricphotogrammetric analysisanalysis ofof calibratedcalibrated images,images, 55 fpsfps videovideo recording,recording, panoramic panoramic mosaic mosaic image,image, andand overlapoverlap withwith measuredmeasured datadata areare knownknown asas “Trimble “Trimble Vision”. Vision”. The The VX VX Spatial Spatial StationStation was was anan RTSRTS with with MagDrive MagDrive servo servo technology, technology, involving involving an an integrated integrated servo/angle servo/angle sensorsensor electromagneticelectromagnetic directdirect drivedrive inin combinationcombination withwith addedadded digitaldigital imagingimaging andand rudi-rudi- mentarymentary scanning scanning capabilities. capabilities. The The scanning scanning function function could could scan scan 20 points 20 points per second per second with thewith ability the ability to colorize to colorize scanned scanned scenes scenes based based on recorded on recorded images images [43]. [43]. InIn 2009, 2009, Pentax Pentax presented presented an RTSan RTS with with integrated integrated cameras cameras into its into instruments, its instruments, known asknown Visio as series Visio model series R-400VDN. model R-400VDN. The camera The wascamera placed was onplaced the telescope, on the telescope, and a screen and a displayingscreen displaying captured captured images andimages video and from video the from camera the was camera also placedwas also on placed the telescope on the abovetelescope the above ocular the (Figure ocular8). (Figure The FOV 8). The was FOV 8.8 was◦ × 8.8°8.8◦ ×, and8.8°, itand could it could record record video video at 10at fps10 fps [44 ].[44]. LeicaLeica introducedintroduced itsits IATSIATS instrument instrument in in 2010 2010 called called Viva;Viva; two two models models with with overview overview camerascameras placedplaced onon thethe telescopetelescope werewere presented:presented: LeicaLeica TS11TS11 andand LeicaLeica TS15TS15 (Figure(Figure8 ).8). ModelModel TS15 TS15 was was motorizedmotorized using using a a servofocusservofocus drive drive withwith additionaladditional sensors sensors for for automatic automatic andand remoteremote control,control, unlikeunlike TS11.TS11. TheThe overviewoverviewcamera camerahad hadan anoptical opticalaxis axisdifferent different from from thethe opticaloptical axisaxis ofof the the telescope, telescope, which which resulted resulted inin parallax; parallax; thus,thus, the the center center of of the the optical optical axisaxis ofof thethe telescopetelescope diddid notnot coincidecoincide withwith thethecenter centerof of the the image image sensor. sensor. TheTherelative relative positionposition ofof thesethese twotwo axesaxes waswas determineddetermined byby calibrationcalibration atat thethe timetime ofof itsits manufacturing.manufacturing. TheThe effecteffect of of parallax parallax could could also also be be corrected corrected by by measuring measuring the the distance distance to to the the target. target. The The integratedintegrated cameras cameras in in both both models models had had a a 5 5 MPx MPx CMOSCMOS image image sensor, sensor, with with fixed fixed focus focus and and ◦ ◦ anan FOVFOV ofof 15.515.5°× × 11.7°,11.7 , with with the the possibility possibility of recording video at 20 fps [[45,46].45,46].

Figure 8. First commercial IATS from Topcon (GPT-7000i) [41], from Trimble (VX Spatial Station) Figure 8. First commercial IATS from Topcon (GPT-7000i) [41], from Trimble (VX Spatial Station) [43], [43], from Pentax (Visio model R-400VDN) [44], and from Leica Viva (TS11/TS15) [45]. from Pentax (Visio model R-400VDN) [44], and from Leica Viva (TS11/TS15) [45]. It should also be mentioned that, at the beginning of this commercial period when IATS systems were brought to the geodetic scene, the first IATS instrument was actually presented by Sokkia (SET3110MV) at INTERGEO 2002. The instrument had two inte- grated cameras, an overview camera on the telescope and another camera integrated in Appl. Sci. 2021, 11, 3893 11 of 29

Appl. Sci. 2021, 11, x FOR PEER REVIEW 11 of 29

It should also be mentioned that, at the beginning of this commercial period when IATS systems were brought to the geodetic scene, the first IATS instrument was actually presentedthe telescope. by Sokkia The (SET3110MV) curious aspect at INTERGEO of this instrument 2002. The was instrument that it did had not two have integrated an ocular; cameras,instead, an the overview expert performed camera on measurements the telescope using and anothera remote camera controller integrated displaying in the the live telescope.video obtained The curious from aspect the integrated of this instrument cameras. was that it did not have an ocular; instead, the expertIn performed2013, Leica measurements introduced the using new aseries remote called controller Nova displayingwith two models the live TS50 video and obtainedMS50. from The theinstruments, integrated in cameras. addition to an overview camera, had an integrated camera in theIn telescope 2013, Leica with introduced an FOV of the 1.3° new × 1.0°, series an called8× digital Nova zoom with of twothe overview models TS50 camera, and and MS50.a 30× The optical instruments, zoom of inthe addition telescope to camera, an overview as well camera, as an 8× had digital an integrated zoom. That camera way, the ◦ ◦ in theimages telescope from withthe telescope an FOV of camera 1.3 × were1.0 , magn an 8×ifieddigital 30×. zoom Leica of MS50 the overview also had camera,a scanning andfunction a 30× optical with the zoom ability of theto scan telescope 1000 points camera, per as second. well as Leica an 8 ×TS50digital did not zoom. have That a scan- way,ning the function. images from Both themodels telescope used camerathe same were motorization magnified using 30× .a Leicaservofocus MS50 drive. also had On the a scanningbasis of functionthe scanning with function the ability with to video scan 1000and photo points capture per second. capabilities, Leica TS50 a combined did not ap- haveproach a scanning to image function. analysis Both and modelsscanning used was thedeveloped same motorization [47,48]. In 2015, using Trimble a servofocus introduced drive.its state-of-the-art On the basis of IATS the scanningmodel S9 functionwhich used with the video same and camera photo and capture image capabilities,sensor as Trim- a combinedble VX Spatial approach Station to imageintroduced analysis in 2007. and However, scanning wasthe S9 developed comes in [at47 ,least48]. 10 In 2015,different Trimblepossible introduced configurations. its state-of-the-art The S9 replaced IATS modelthe VX S9 and which S8 for used both the precise same cameraand long-range and imageapplications. sensor as TrimbleIt offered VX higher Spatial angular Station and introduced distance in accuracy 2007. However, than the the VX S9 Spatial comes Station in at least[49]. 10 The different predecessor possible ofconfigurations. today’s state-of-the-art The S9 replaced IATS and the IASTS VX and company, S8 for both Topcon, precise un- andfortunately long-range abandoned applications. itsIt original offered design higher with angular twoand integrated distance cameras accuracy in thanthe instrument; the VX Spatial Station [49]. The predecessor of today’s state-of-the-art IATS and IASTS company, in 2014, it offered IATS model DS-200i with one camera, an overview ultra-wide camera Topcon, unfortunately abandoned its original design with two integrated cameras in the on the telescope with a 5 MPx image sensor, which was mainly integrated for providing instrument; in 2014, it offered IATS model DS-200i with one camera, an overview ultra- imaging documentation and a live video stream on the remote controller [50]. wide camera on the telescope with a 5 MPx image sensor, which was mainly integrated for In Table 1, Leica TS60, Trimble S9, and Topcon DS-200i are compared with their main providing imaging documentation and a live video stream on the remote controller [50]. specifications and characteristics. These instruments do not offer a scanning function as In Table1, Leica TS60, Trimble S9, and Topcon DS-200i are compared with their main they are classified as total stations not multi-stations, although S9 offers a combined model specifications and characteristics. These instruments do not offer a scanning function as with a scanning function. they are classified as total stations not multi-stations, although S9 offers a combined model with a scanning function. Table 1. Main specifications of currently available IATS [49–51].

Specifications Table 1. Main specifications Leica TS60 of currently available Trimble IATS [49 S9–51 ]. Topcon DS-200i Camera/sensor 2×/CMOS 1×/CMOS 1×/CMOS Specifications Leica TS60 Trimble S9 Topcon DS-200i Resolution 5 MPx 3 MPx 5 MPx Camera/sensor 2×/CMOS 1×/CMOS 1×/CMOS Fps 20 Hz 5 Hz Live video1 Resolution 5 MPx 3 MPx 5 MPx FOV overview/telescopeFps 15.5°20 Hz× 11.7°/1.3° × 1.0° 16.5° 5 Hz × 12.3°/✗ LiveUltra-wide video 1 1/✗ FOVZoom overview/telescope overview/telescope 15.5◦ × 11.7◦/1.38×/30×◦ × 1.0 ◦ 16.5◦ × 12.38×/◦/✗ Ultra-wide 11/ ZoomAccuracy overview/telescope distance (prism) 8×0.6/30 mm× + 1 ppm 0.88× /mm + 1 ppm 1.5 1mm + 2 ppm AccuracyAccuracy distance distance (prism) (non-prism) 0.6 mm2 +mm 1 ppm + 2 ppm 0.8 mm 2 mm + 1 ppm + 2 ppm 1.5 mm 2 mm + 2 ppm+ 2 ppm Accuracy distance Accuracy Hz and V 2 mm + 2 ppm0.5″ 2 mm + 2 ppm0.5″ 2 mm + 2 ppm1″ (non-prism) AccuracyScanning Hz and function V 0.500 ✗ 0.500 • 100 ✗ ScanningYear function released 2015 • 2015 2014 Year released 1 Specification2015 is not available; ✗—not integrated; 2015 •—optional. 2014

1 Specification is not available; —not integrated; •—optional. Today, Leica offers the models TS60 and MS60 from the Nova series introduced for the first time in 2015 with improved functions of ATR (ATR plus), reflector self-search Today, Leica offers the models TS60 and MS60 from the Nova series introduced for the using the PowerSearch function, dynamic look function, motorized direct drives based on first time in 2015 with improved functions of ATR (ATR plus), reflector self-search using Piezo technology, improved scanning function with 3D laser scanning of 30,000 points per the PowerSearch function, dynamic look function, motorized direct drives based on Piezo second, image-assisted surveying, and documentation using advanced artificial intelli- technology, improved scanning function with 3D laser scanning of 30,000 points per second, image-assistedgence; it is called surveying, the first and self-learning documentation multi-station using advanced [51]. artificial intelligence; it is called theTrimble first self-learning introduced multi-stationits new generation [51]. of IATS instruments in 2016, model SX10. This isTrimble the first introduced commercial its TS, new i.e., generation IATS without of IATS aninstruments ocular. Three in 5 2016, MPx model cameras SX10. (overview, This is theprimary, first commercial and general) TS, are i.e., integrated IATS without in the an telescope, ocular. Three offering 5 MPx a maximum cameras (overview, total FOV of primary,360° × and 300° general) with 84× are zoom. integrated The general in the telescope, camera enables offering an a maximumFOV of 57.5° total × 43.0°. FOV ofAn 360 expert◦ controls the instrument using a remote controller via live video at 15 fps. SX10 combines high-density 3D scan data, enhanced imaging called Trimble Vision, and high-accuracy total station data. It uses MagDrive servo technology, and it can measure dense 3D scan Appl. Sci. 2021, 11, 3893 12 of 29

Appl.Appl.Appl. Sci. Sci. Sci. 2021 2021 2021, 11, 11, 11, x, x, FORx FOR FOR PEER PEER PEER REVIEW REVIEW REVIEW 121212 of of of29 29 29 × 300◦ with 84× zoom. The general camera enables an FOV of 57.5◦ × 43.0◦. An expert controls the instrument using a remote controller via live video at 15 fps. SX10 combines high-density 3D scan data, enhanced imaging called Trimble Vision, and high-accuracy totaldatadatadata stationat at at up up up to to to data.26,600 26,600 26,600 It points usespoints points MagDrive per per per second second second servo with with with technology,high high high precision precision precision and over over over it can the the themeasure full full full measurement measurement measurement dense 3D range scanrange range dataofofof up up atup to upto to 600 600 to600 26,600m m m [52]. [52]. [52]. points There There There per was was was second some some some with confus confus confus highion,ion,ion, precision even even even among among overamong the Trimble Trimble Trimble full measurement developers, developers, developers, range as as as to to to ofwhetherwhetherwhether up to 600it it itshould should mshould [52 ].be be be Thereconsidered considered considered was some a a newa new new confusion, scanner scanner scanner or or evenor a a newa new new among total total total station. Trimble station. station. However, developers,However, However, it it itis asis isstill still tostill whetherclassifiedclassifiedclassified it as shouldas as a a TSa TS TS because bebecause because considered of of of its its its characteristic characteristic acharacteristic new scannerss regardings regarding orregarding a new the totalthe the telescope, telescope, telescope, station. However,angle angle angle and and and itdistance distance distance is still classifiedmeasurements,measurements,measurements, as a TS and and becauseand the the the possibility possibility ofpossibility its characteristics to to to measure measure measure regarding ex ex explicitlyplicitlyplicitly the one one one telescope, point. point. point. Since Since Since angle it it itcombines andcombines combines distance im- im- im- Appl. Sci. 2021, 11, x FOR PEER REVIEW 11 of 29 measurements,agingagingaging with with with scanning scanning scanning and theon on on a possibilitya levela level level that that that tois is isnot measurenot not rudimentary, rudimentary, rudimentary, explicitly we we onewe can can can point. call call call it it Sinceitthe the the first first first it combinesimage-as- image-as- image-as- imagingsistedsistedsisted scanning scanning scanning with scanning total total total station station station on a (IASTS). level(IASTS). (IASTS). that At At At is the the notthe beginning beginning rudimentary,beginning of of of this this this we year, year, canyear, Trimble callTrimble Trimble it the introduced introduced introduced first image- its its its assistedsuccessorsuccessorsuccessor scanning SX12. SX12. SX12. The The totalThe technical technical technical station (IASTS).data data data offered offered offered At the at at at the beginningthe the moment moment moment of describe thisdescribe describe year, the the Trimblethe following following following introduced upgrades: upgrades: upgrades: its successor8.18.18.1 MPx MPx MPx image image SX12. image sensors, Thesensors, sensors, technical 107× 107× 107× zoom datazoom zoomthe offered of of of telescope.the the the main atmain main the camera, The momentcamera, camera, curious and and describeand a aaspect newa new new the laser laser oflaser following this pointer, pointer, pointer, instrument upgrades: while while while the wasthe the that it did not have an ocular; 8.1remainingremainingremaining MPx image features features features sensors, regarding regarding regarding 107× zoom angle angle angleinstead, of and theand and mainthedistance distance distance expert camera, measurements measurements performedmeasurements and a new measurements laserand and and scanning pointer,scanning scanning using while function function function a theremote controller displaying the live remainingareareare at at at the the the same featuressame same level level level regarding as as as SX10 SX10 SX10 angle[53]. [53]. [53].video and distanceobtained measurements from the integrated and scanning cameras. function are at theTopconTopcon sameTopcon level presented presented presented as SX10 its its [its 53IASTS IASTS IASTS]. GTL-1000 GTL-1000 GTL-1000In 2013, in in in 2019 2019 Leica2019, ,and ,and andintroduced it it itis is isthe the the world’ world’ world’the newss firsts first first series RTS RTS RTS and andcalled and laser laser laser Nova with two models TS50 and scannerscannerscannerTopcon with with with presented an an an inbuilt inbuilt inbuilt its camera, camera, camera, IASTS which GTL-1000which MS50.which is is Theisprimarily primarily primarily in instruments, 2019, andused used used it for for is forin thecolorizing colorizingaddition colorizing world’s tothe firstthe thean scanned scanned RTSscannedoverview and scenes. scenes. scenes. laser camera, had an integrated camera in scannerTheTheThe camera camera camera with has has an has a inbuilt a 5a 5 MPx5 MPx MPx camera, image image image sensor whichsensor sensorthe telescope which iswhich which primarily is is isalsowith also also used used usedan used FOV forfor for for colorizinglive oflive live 1.3° video. video. video. × 1.0°, the A A A3D scanned 3D an3D laser laser8× laser digital scanner scenes.scanner scanner zoom of the overview camera, and Theisis isinstalled installed camerainstalled hasin in in place aplace place 5 MPx of of of the imagethe the RTS RTS RTS sensor handle, handle,a handle, 30× which optical resultin resultin resultin is alsozoomgg gin usedin in ofa a asolution the forsolutionsolution livetelescope video.that that that it camera,it Aitis is 3Disboth both both laser as a a awellgeodetic scannergeodetic geodetic as an 8× digital zoom. That way, the ismeasuringmeasuringmeasuring installed station instation station place and and ofand thea a geodetica geodetic RTSgeodetic handle,images 3D 3D 3D laser laser laser resulting from sca sca sca nner.thenner.nner. intelescope The aThe The solution instrument instrument instrument camera that ituseswere uses uses is both direct magndirect direct a ified geodeticdrive drive drive 30×. via via via Leica MS50 also had a scanning measuringananan ultrasonic ultrasonic ultrasonic station motor. motor. motor. and The The The a geodeticmain main main advantage advantage advantagefunction 3D laser withof scanner.of of this this this the IASTS IASTS IASTSability The compared instrument compared comparedto scan 1000 to to usesto other other pointsother direct commercially commercially commercially per drive second. via Leica TS50 did not have a scan- anavailableavailableavailable ultrasonic IASTS IASTS IASTS motor. is is isthe the the The scan scan scan main rate, rate, rate, advantage which ningwhich which function. is is ison ofon on thisthe the the Both levelIASTS level level models of of comparedof today’s today’s today’s used terrestrial terrestrial toterrestrialthe other same commerciallylaser lasermo lasertorization scanners scanners scanners using a servofocus drive. On the availableandandand is is isspecified specified specified IASTS isas as as the100,000 100,000 100,000 scan rate,points points points whichbasis per per per second. of issecond. second. onthe thescanning It It levelItis is isthe the the of functionfirst first today’sfirst such such such with terrestrialinstrument instrument instrument video laseran in ind in the photo scannersthe the world world world capture capabilities, a combined ap- and[54].[54].[54]. is specified as 100,000 points perproach second. to image It is the analysis first such and instrument scanning was in the developed world [54 [47,48].]. In 2015, Trimble introduced InInInIn TableTable Table Table2 2,, 2, Leica 2,Leica Leica Leica MS60, MS60, MS60, MS60, Trimble Trimble Trimble Trimbleits SX12, state-of-the-artSX12, SX12, SX12, and an an andd Topcon dTopcon Topcon Topcon IATS GTL-1000 GTL-1000 GTL-1000 GTL-1000model S9 are are are whichare compared compared compared compared used with thewith with with same their their their their camera and image sensor as Trim- mainmainmainmain specificationsspecifications specifications specifications andand and and characteristics.characteristics. characteristics. characteristics.ble VX AllAll All SpatialAll instrumentsin in instrumentsstrumentsstruments Station havehave introducedhave have aa a scanning scanninga scanning scanning in 2007. functionfunction function function However, andand and and theythey they theythe S9 comes in at least 10 different areareareare classifiedclassified classified classified asas as as multi-stations,multi-stations, multi-stations, multi-stations, so-calledso-called so-called possibleso-called IASTS.IASTS. IASTS. configurations.IASTS. TheseThese These These threethree three three The instruments,instruments, instruments, instruments,S9 replaced atat atthe at thethe the theVX moment,moment, moment, moment,and S8 for both precise and long-range representrepresentrepresentrepresent thethe the the state-of-the-artstate-of-the-art state-of-the-art state-of-the-art geodeticgeo geo geodeticapplications.deticdetic instrumentsinstruments instruments instruments It offered inin in in thethe the the world.higherworld. world. world. angular and distance accuracy than the VX Spatial Station [49]. The predecessor of today’s state-of-the-art IATS and IASTS company, Topcon, un- TableTableTableTable 2.2. 2. Main2.Main Main Main specificationsspecifications specifications specifications ofof of currentlyofcurrently currently currentlyfortunately availableavailable available availableIASTS IASTSabandoned IASTS IASTS [51[51,53,54]. [51,53,54]. [51,53,54].,53,54 its]. original design with two integrated cameras in the instrument; in 2014, it offered IATS model DS-200i with one camera, an overview ultra-wide camera SpecificationsSpecificationsSpecificationsSpecifications Leica Leica Leica Leica MS60 MS60 MS60 Trimble Trimble Trimble SX SXSX SX 12 12 12 Topcon Topcon Topcon GTL-1000 GTL-1000 GTL-1000 on the telescope with a 5 MPx image sensor, which was mainly integrated for providing imaging documentation and a live video stream on the remote controller [50]. In Table 1, Leica TS60, Trimble S9, and Topcon DS-200i are compared with their main specifications and characteristics. These instruments do not offer a scanning function as they are classified as total stations not multi-stations, although S9 offers a combined model with a scanning function.

Table 1. Main specifications of currently available IATS [49–51].

Specifications Leica TS60 Trimble S9 Topcon DS-200i Camera/sensor 2×/CMOS 1×/CMOS 1×/CMOS

Resolution 5 MPx 3 MPx 5 MPx Camera/sensor 2×/CMOS 3×/CMOS 1×/CMOS CameraCameraCamera/sensor/sensor/sensor 2×/CMOS2×/CMOS2×/CMOSFps 3×/CMOS3×/CMOS3×/CMOS20 Hz 1×/CMOS1×/CMOS1×/CMOS 5 Hz Live video1 Resolution 5 MPx 8.1 MPx 5 MPx ResolutionResolutionResolution 55 MPx5 MPx MPx 8.1 8.1 8.1 MPx MPx MPx 5 5 MPx 5 MPx MPx 1 Fps FOV overview/telescope20 Hz 15.5° 15× 11.7°/1.3° Hz × 1.0° Live16.5° video × 12.3°/✗ Ultra-wide /✗ FpsFps ◦ 2020 Hz Hz◦ ◦ ◦ 15 15 Hz Hz◦ ◦ Live Live video video 1 FOV overview/telescopeFps Zoom15.5 × overview/telescope11.720 Hz/1.3 × 1.0 2 Total 360 15 8×/30×Hz× 300 270◦ Live× 360 video8×/◦/✗ 2 2 FOVFOVZoomFOV overview/telescope overview/telescope overview/telescope 15.5°Accuracy15.5°15.5° × × 11.7°/1.3°× 11.7°/1.3° 11.7°/1.3°8× distance/30× × × 1.0°× 1.0°(prism) 1.0° Total2Total Total0.6 107360° 360° 360°mm× ×3 × +300°× 300° 1300° ppm 270°270°270° 0.8 × × 1mm360°/× 360°/ 360°/ +✗ ✗1✗ ppm 1.5 mm + 2 ppm 3 1 ZoomZoomZoomAccuracy overview overview overview distance/telescope/telescope/telescope (prism) Accuracy1 mm 8×/30×distance8×/30×8×/30× + 1.5 ppm (non-prism) 1 mm107×2107× 107×mm + 1.5 3 + ppm3 2 ppm 1 mm 2 mm +1 21 ppm + 2 ppm 2 mm + 2 ppm Accuracy distance (non-prism) 2 mm + 2 ppm 2 mm + 1.5 ppm 2 mm + 2 ppm AccuracyAccuracyAccuracy distance distance distance (prism) (prism) (prism) 11Accuracy mm1 mm mm + + 1.5+ 1.5 1.5 Hzppm ppm ppm and V 1 1 mm 1 mm mm + + 1.5+ 1.5 1.50.5 ppm ppm ppm″ 1 1 mm 1 mm mm + + 2+ 2 ppm0.52 ppm ppm″ 1″ Accuracy Hz and V 100 100 100 AccuracyAccuracyAccuracy distance distance distance (non-prism) (non-prism) (non-prism) 22 mm2 mm mm + + 2+ 2 ppm2 ppm ppm 2 2 mm 2 mm mm + + 1.5+ 1.5 1.5 ppm ppm ppm 2 2 mm 2 mm mm + + 2+ 2 ppm2 ppm ppm Scan rate Scanning30,000 Hzfunction 26,600 Hz✗ 100,000 Hz• ✗ Accuracy Hz and V AccuracyAccuracyYear released Hz Hz and and V V Year120201″ 1″ released″ 202111″1″ ″ 2015 201911″1″ ″ 2015 2014 ScanScanScan rate rate rate 30,00030,00030,000 Hz Hz Hz 1 26,600 26,600 26,600 Hz Hz Hz 100,000 100,000 100,000 Hz Hz Hz 1 Specification is not available; 2 FOV using all three cameras;Specification3 main camera is zoom;not available;—not integrated. ✗—not integrated; •—optional. YearYearYear released released released 202020202020 202120212021 201920192019 1 1Specification 1Specification Specification is is isnot not not available; available; available; 2 2FOV 2FOV FOV using using using all all all three three three cameras; cameras; cameras;Today, 3 3main 3main main cameraLeica camera camera offers zoom; zoom; zoom; the ✗ ✗ —✗— models—notnotnot integrated. integrated. integrated. TS60 and MS60 from the Nova series introduced for the first time in 2015 with improved functions of ATR (ATR plus), reflector self-search AllAllAll currently currently currently available available available commercial commercial commercialusing theIATS IATS IATS PowerSearch and and and IASTS IASTS IASTS instrumentsfunction, instruments instruments dynamic use use use recorded recorded recorded look function, images images images motorized direct drives based on andandand videos videos videos for for for additional additional additional attributes attributes attributesPiezo of of of the technology,the the measured measured measured improved point, point, point, i.e., i.e., i.e., scanning primarily primarily primarily function for for for documenta- documenta- documenta- with 3D laser scanning of 30,000 points per tiontiontion and and and colorizing colorizing colorizing the the the scanned scanned scanned scenes. scenes.second, scenes. Stan Stan Stanimage-assisteddarddarddard measurement measurement measurement surveying, methods methods methods and in documentationin in the the the field field field have have have using advanced artificial intelli- gence; it is called the first self-learning multi-station [51]. Trimble introduced its new generation of IATS instruments in 2016, model SX10. This is the first commercial TS, i.e., IATS without an ocular. Three 5 MPx cameras (overview, primary, and general) are integrated in the telescope, offering a maximum total FOV of 360° × 300° with 84× zoom. The general camera enables an FOV of 57.5° × 43.0°. An expert controls the instrument using a remote controller via live video at 15 fps. SX10 combines high-density 3D scan data, enhanced imaging called Trimble Vision, and high-accuracy total station data. It uses MagDrive servo technology, and it can measure dense 3D scan Appl. Sci. 2021, 11, 3893 13 of 29

All currently available commercial IATS and IASTS instruments use recorded images and videos for additional attributes of the measured point, i.e., primarily for documentation and colorizing the scanned scenes. Standard measurement methods in the field have been largely developed and improved by technological developments of TS. Manual focusing has been replaced by auto-focusing; moreover, using remote control, an expert can select the point of interest defined by a pixel from a live video/image on a display, thereby pointing the instrument to a signalized or unsignalized point. For pointing the instrument to the point of interest and for tracking the reflector in the field, instruments use motorization, which is currently realized by piezo, magnetic, or ultrasonic drives depending on the manufacturer. The recorded images can be automatically downloaded for each point or for a set of points for documentation purposes. This way, it is possible to visualize these points. More importantly, we can perform photogrammetric processing of collected georeferenced images afterward and, e.g., use them for structural and geo-monitoring, leveling, and fusion of laser scan and image data for deformation monitoring. However, commercial IATS and IASTS instruments still do not use the full potential and functions of these instruments [8]. In the next section, the potential for application of the latest state-of-the-art IATS is presented.

3. Measurement and Processing Approaches Depending on the developed hardware and software, integrated sensors, and different functions through the implementation of various algorithms in TS, we can distinguish types of TS (Table3).

Table 3. Different types of TS according to integrated sensors and functions.

TS RTS IATS IASTS EDM     Reflectorless ••   EDM Data registration     Motorization     Image sensor  /•   Automated  /•   aiming Tracking  /•   Imaging     Scanning   /•  —integrated; —not integrated; •—optional.

State-of-the-art IATS and IASTS represent a new kind of RTS, unifying the geodetic precision of TS with the areal coverage of images, as well as laser scans. IATS offer the user an image-capturing system with integrated camera(s) using image sensor(s) in addition to a polar 3D point measurement system. IASTS are equipped with a scanning function with lower scan rates comparable to classical terrestrial laser scanners. The greatest advantage of using such a (multi-sensor) instrument is the common coordinate system of all built-in sensors, as long as appropriate a priori calibration is provided [55]. These multi-sensor systems can capture images for documentation and for pho- togrammetric measurements. They can even merge more of these captured images into a panoramic image mosaic using camera rotation, i.e., telescope rotation. With appropriate calibration, these images are accurately georeferenced and oriented, such that they can be immediately used for direction measurements with no need for object control points or further photogrammetric orientation processes [15]. Appl. Sci. 2021, 11, 3893 14 of 29

3.1. IATS Basic Principles The combination of cameras with image sensors and theodolites was described long ago by [29,56]. In this section, we explain the basic working principles of state-of-the-art Appl. Sci. 2021, 11, x FOR PEER REVIEWIATS Leica MS50, which are also applicable to MS60 and other IATS produced by manufac-14 of 29 turers with two cameras (overview and telescope). Overview and telescope cameras offer the possibility to record high-resolution images or panoramic image mosaics, which can be directly linked and referenced to the 3D point measurement system. These images can can be directly linked and referenced to the 3D point measurement system. These images be used for field documentation and, most importantly, for a posteriori photogrammetric can be used for field documentation and, most importantly, for a posteriori photogram- processing. The photogrammetric resolution of 1 px of the telescope camera corresponds metric processing. The photogrammetric resolution of 1 px of the telescope camera corre- to 5 cc, which results in a resolution of 2 mm at 200 m [57]. The expert uses the overview sponds to 5 cc, which results in a resolution of 2 mm at 200 m [57]. The expert uses the camera for roughly aiming at the target, signalized or unsignalized. The overview camera overview camera for roughly aiming at the target, signalized or unsignalized. The over- image gives an overview of the surveying area; alternatively, a live video can be observed view camera image gives an overview of the surveying area; alternatively, a live video on instrument display. Using the telescope camera, which is located on the optical axis, can be observed on instrument display. Using the telescope camera, which is located on a telescope magnification of 30× is at the expert’s disposal for very precise target aiming the optical axis, a telescope magnification of 30× is at the expert’s disposal for very precise (Figure9). target aiming (Figure 9).

FigureFigure 9. 9.Leica Leica MS50 MS50 schematic schematic cross-sectional cross-sectional view view of of the the telescope telescope with with image image sensoros sensoros and and processor processor integration, integration, as wellas aswell motorization as motorization for autofocus; for autofocus; scheme scheme modifed modifed for this for studythis study from from [57]. [57].

The achievable accuracy of of angular angular measurem measurementsents using using this this camera camera is is 1”. 1”. Using Using the the integratedintegrated imageimage sensor,sensor, singlesingle autofocusautofocus andand continuouscontinuous autofocusautofocus functions functions are are possible. possi- Integrationble. Integration of the of image the image sensor sensor is done is done in such in such way thatway athat focused a focused image image is simultaneously is simulta- visibleneously on visible the instrument on the instrument display display and on and the instrumenton the instrument optical optical path through path through the ocular. the Aocular. Porro A prism Porro reflects prism thereflects incoming the incoming light through light through the telescope the telescope optics onto optics the onto imaging the CMOSimaging sensor CMOS and sensor directly and onto directly the ocularonto the for ocular visual for aiming visual if aiming needed. if The needed. focusing The modulefocus- productioning module guaranteesproduction aguarantees 1” angular a measurement1” angular measurement accuracy. Adjustment accuracy. Adjustment and calibration and processescalibration are processes key factors are tokey achieve factors the to highestachieve qualitythe highest of the quality imaging of the functionality. imaging func- tionality.Prior to the available IATS, there were RTS, which, with implemented ATR functions, are alsoPrior IATS to the in available the narrowest IATS, sensethere were since RTS, image which, sensors with or implemented segmented photodiodesATR functions, are usedare also for IATS signal in target the narrowest detection sense and aiming.since image However, sensors these or segmented image sensors photodiodes could not are be used for imagesignal target documentation detection and aiming photogrammetric. However, measurementsthese image sensors because could the not expert be didused not for have image access documentation to the images and and photogra becausemmetric only the measurements light in the spectrum because measurement the expert signaldid not was have detected access to bythe imagingimages and sensors becaus ore only photodiodes the light in [58 the]. spectrum The difference measurement today is evidentsignal was in the detected way in by which imaging the expertsensors extracts or photodiodes the information [58]. The from difference the integrated today camera,is evi- i.e.,dent from in the a way single in imagewhich the (point expert of time)extracts or fromthe information a video (sequence from the ofintegrated images). camera, We can i.e., from a single image (point of time) or from a video (sequence of images). We can also distinguish static or kinematic processes. With modern digital imaging sensors, both measuring tasks are possible and no differentiation of the instrument is necessary [8].

Appl. Sci. 2021, 11, 3893 15 of 29

also distinguish static or kinematic processes. With modern digital imaging sensors, both measuring tasks are possible and no differentiation of the instrument is necessary [8].

3.2. IATS System and Camera Calibration The use of an appropriately calibrated camera is of crucial importance for displacement monitoring. It allows a two-dimensional image coordinate to be presented in angular units, i.e., field angle in the object space, which can then be expressed in linear units. All intrinsic and extrinsic parameters must be known for each captured image, which means that the images are directly georeferenced. They can be used for displacement monitoring with no need for object control points or further photogrammetric orientation processes. Photogrammetric image measurement methods, in order to detect signalized and unsignalized targets, can be combined with functions of the TS to generate precise angle and distance measurements [8]. In this section, we explain the camera calibration placed in the IATS telescope, i.e., in the optical path of the telescope. We focus on this camera and not the overview camera, because it is better to use the telescope camera for the purpose of displacement monitoring since we need highly accurate data for the determination of dynamic displacements with high speed and small amplitudes. Highly accurate data can be obtained using a telescope camera, because it uses telescope optics and magnification; however, we must also perform camera calibration, which involves the determination of intrinsic camera parameters. The intersection of the optical axis with the image sensor plane, i.e., principal point image coordinates, the focal length in the horizontal and vertical directions, and an arbitrary number of coefficients to compensate for radial and tangential lens distortions are intrin- sic parameters which must be determined during camera calibration [59]. The distance between the camera center and the principal point is the focal length of pinhole cameras integrated in today’s IATS. Two focal lengths (horizontal and vertical) must be determined because the pixels on the image sensor are non-square for some cameras [60]. Imperfect manufacturing of the lens and defects in the assembling of the camera result in radial and tangential lens distortions of the linear model for projecting points in the real world to the image plane [60]. Since monitoring assumes measurements between two or more different series or epochs of point measurements, the constant image coordinates of the principal point are void and, for this reason, they do not need to be known in advance. The distortion effects exhibit a spatial regularity, such that neighboring pixels are affected similarly. Thus, dis- tortion effects are reduced to an insignificant extent when computing differences between objects located in similar regions on the image sensor. The frequencies included in a time series, which can be computed via Fourier analysis, are independent of the physical unit of the time series. This is due to the linear property of the Fourier transform (Equation (1)), where a constant factor c may be applied before or after the transformation [59].

F{c f (t)} = cF{ f (t)} . (1)

The frequencies of structure vibrations can be computed from the raw image coordi- nates, i.e., from the image coordinates of measured points. Thus, no camera calibration is needed for this purpose. To present the displacements obtained from two epochs of measurement in linear units (e.g., m, dm, cm, or mm), the camera’s angular resolution must be known. The displacements on the image are determined in pixels and can be computed as angular quantities, which can then be presented as linear units when the measured distance from the instrument to the monitored object is known. The angular resolution expresses the size of one pixel of the image sensor as an angular unit. It presents the angle between two objects that are presented on two neighboring pixels. For a pinhole camera model, the angular resolution α is related to the focal length f by Equation (2).

tan(α) = 1/ f . (2) Appl. Sci. 2021, 11, 3893 16 of 29

To precisely and accurately determine displacements, the camera’s angular resolution must also be precisely and accurately determined, as the object displacements on the image given in pixels must be multiped by the value of angular resolution to be presented in angular units, i.e., in linear units. The object displacements can be up to several pixels; thus, small uncertainty in the value of angular resolution can lead to a large error in the computed angle [59]. Manufacturers support academic research projects covering different calibration meth- ods of integrated cameras. For example, Walser described a camera with an affine chip model and developed a combined approach for taking camera and instrument error cal- ibration into account [11]. Vogel, in contrast, extended the collinearity equations using additional camera parameters to describe the pixel-to-angle relationship in IATS [61]. Knoblach expressed central projection as affine transformation without theodolite errors for the use of different focus lens positions with fewer parameters [13]. The abovemen- tioned methods were compared by Wasmeier [14]. All methods are based on a set of virtual control points for data acquisition during the calibration process. In reality, this set consists of only one “real” point, measured multiple times by rotating the telescope with a camera [8]. When appropriate calibration is provided, the obtained images and video frames from IATS are accurately georeferenced at any time. They are particularly suitable for structural and geo-monitoring, i.e., for objects and Earth surface areas which are very hard to approach. It is also possible to use additional vision technology for leveling applications by reading and analyzing digital leveling staff code patterns [62]. The major advantage of IATS is the fact that the methods and approaches of photogrammetry can be used in combination with the precision distance and angle measurements of TS. Together with the station coordinates and orientation of the instrument, each captured image is directly georeferenced and can be used for direction measurements with no need for object control points or further photogrammetric orientation processes. A universal description of the relationship between a homogeneous 3D coordinate (Equation (3)) and its image projection (Equation (4)) was given by Wagner [55]. The intrinsic camera parameter K and the extrinsic parameters (the rotation matrix R and the translation vector T) are thereby combined into a single, compact 3 × 4 projection matrix P (Equation (5)). T Xe = [X, Y, Z, 1] . (3) T ex = [x, y, 1] . (4) ex = PXe = K[R|T]Xe. (5) The extrinsic parameters R and T shown in Figure 10 are composed of three different Euclidean transformations [63]: 1. Transformation from the world or object coordinate system into the theodolite coordi- nate system (Rw, Tw); 2. Transformation (pure rotation) into the telescope coordinate system (Rt); 3. Transformation into the camera coordinate system (Rc, Tc). Appl.Appl. Sci.Sci.2021 2021,,11 11,, 3893 x FOR PEER REVIEW 1717 ofof 2929

FigureFigure 10.10. Transformations needed to to convert convert image image coordinates coordinates into into theodo theodolitelite angles angles and and vice vice versaversa [[8].8].

3.3.3.3. IATSIATS MeasurmentMeasurment ProceduresProcedures andand DataData ProcessingProcessing ExperimentalExperimental oror inin situsitu measurementmeasurement proceduresprocedures andand datadata processingprocessing workflowworkflow forfor conductingconducting displacementdisplacement andand vibrationvibration monitoringmonitoring (i.e.,(i.e., structuralstructural healthhealth monitoring)monitoring) areare shownshown onon FigureFigure 1111.. TheThe workflowworkflow cancan bebe fullyfully automaticautomatic oror semi-automatic. semi-automatic. TheThe wholewhole processprocess cancan be automated using using Leica’s Leica’s GeoCOM GeoCOM interface. interface. A Auser user interaction interaction is only is only re- requiredquired at atthe the beginning beginning to initially to initially aim aimat the at targets the targets to be tomonitored be monitored and to andset the to setcamera the cameraproperties properties [64]. [64]. AtAt thethe same same time, time, several several point point targets targets can can be be monitored monitored at different at different areas areas of structures of struc- ortures Earth or surfacesEarth surfaces as long as as long they appearas they inappe thear camera’s in the camera’s FOV. The FOV. IATS canThe recordIATS can a video record or image,a video depending or image, depending on the purpose on the of purpose monitoring. of monitoring. If performing If perfor structuralming monitoring structural mon- with theitoring aim with of determining the aim of thedetermining natural frequencies the natural and frequencies dynamic and displacements dynamic displacements of structures (e.g.,of structures during (e.g., load during tests or load everyday tests or traffic), everyday then traffic), videos then are videos recorded. are recorded. If performing If per- theforming geo-monitoring the geo-monitoring of the Earth’s of the surfaceEarth’s surf (e.g.,ace landslides), (e.g., landslides), then images then images are recorded are rec- periodically. An expert must set up the IATS and aim at the target, as well as set up the orded periodically. An expert must set up the IATS and aim at the target, as well as set up camera parameters. For long-term monitoring where targets are observed periodically, the camera parameters. For long-term monitoring where targets are observed periodi- in the first epoch of measurements, the expert must aim at the target and, in subsequent cally, in the first epoch of measurements, the expert must aim at the target and, in subse- epochs of measurements, the registered positions of targets from the first epoch are used quent epochs of measurements, the registered positions of targets from the first epoch are for automated aiming. used for automated aiming. The movements between targets, represented by their pixels on images, can be com- The movements between targets, represented by their pixels on images, can be com- puted relative to the first frame/epoch or between consecutive frames/epoch. The latter is puted relative to the first frame/epoch or between consecutive frames/epoch. The latter is not appropriate for structural health monitoring when recording video, because the pixel not appropriate for structural health monitoring when recording video, because the pixel movements relative to the first frame must be computed by cumulating the consecutive movements relative to the first frame must be computed by cumulating the consecutive movements between each frame, which results in the propagation of random errors in movements between each frame, which results in the propagation of random errors in consecutive movements. For vibration monitoring, i.e., determination of a structure’s naturalconsecutive frequencies, movements. a conversion For vibration from monitori pixel movementsng, i.e., determination to linear units of a is structure’s not necessary. nat- Toural eliminate frequencies, trends a conversion in the time from series pixel prior movements to the computation to linear units of theis not frequencies necessary. via To Fouriereliminate transformation, trends in the time accelerations series prior are to computed the computation from the of pixel the movements.frequencies via The Fourier usage oftransformation, consecutive pixel accelerations movements are iscomputed preferable from to reducethe pixel noise movements. in the resulting The usage frequency of con- spectrumsecutive pixel [59]. movements is preferable to reduce noise in the resulting frequency spec- trumPossible [59]. IATS telescope rotations which result in camera rotations must also be consid- ered forPossible highly IATS accurate telescope displacement rotations monitoring. which result These in camera small camerarotations rotations must also can be occur con- unintentionallysidered for highly ordue accurate to tilts displacement of the whole station.monitoring. Both These effects small are measured camera rotations by IATS, i.e.,can angleoccur andunintentionally tilt readings. or Correction due to tilts is achievedof the whole by subtracting station. Both the effects camera are rotations measured from by theIATS, image-based i.e., angle and angle tilt measurements readings. Correction to the target. is achieved by subtracting the camera rota- tions from the image-based angle measurements to the target. Appl. Sci. 2021, 11, 3893 18 of 29 Appl. Sci. 2021, 11, x FOR PEER REVIEW 18 of 29

FigureFigure 11. 11.IATS IATS measurementsmeasurements andand datadata processingprocessing workflowworkflow forfor conducting displacement displacement and and vibration vibration monitoring monitoring (structural(structural health health monitoring), monitoring), modified modified forfor thisthis studystudy fromfrom [[64].64].

UsingUsing the proposed image-based measurements, measurements, the the detection detection of of object object movements movements perpendicularperpendicular to to the the IATS IATS line line of ofsight sight is possible. is possible. To measure To measure movements movements along the along sight- the sightinging axis, axis,the IATS the IATS EDM EDM can be can used; be used; alternatively, alternatively, if available if available IATS IATS scanner scanner data data can canbe beused, used, i.e., i.e., fusion fusion of laser of laser scans scans and andcamera camera data datato RGB to RGB+ D images. + D images. This approach This approach com- combinesbines the advantages the advantages of two of methods. two methods. The distance The distance of the laser of the scan laser dense scan point dense clouds point cloudschanges changes as a function as a function of line of of sight line of and sight can and be easily can be detected. easily detected. In contrast, In contrast, camera image camera imagedata are data most are sensitive most sensitive to displacements to displacements perpendicular perpendicular to the line of to sight the (i.e., line to of the sight view- (i.e., toing the direction viewing of directionthe telescope of the in which telescope the camera in which is integrated). the camera This is integrated). way, displacements This way, displacementscan be determined can be by determined extracting bycommon extracting image common features image (RGB features channel) (RGB and channel) tracking and trackingthem over them a series over aof series monitoring of monitoring measurem measurementent epochs. epochs.In combination In combination with the with depth the Appl. Sci. 2021, 11, 3893 19 of 29

depth channel (D), the determined image features (from RGB + D) represent 3D coordinates, which can be used for the computation of 3D displacement vectors [55]. For the monitoring applications described in this paper, only the angular resolution of the camera needs to be calibrated. This can be done with a fast and simple approach and without additional measurement equipment by using the total station’s ability to automatically rotate its telescope to precisely known positions [59].

4. Application of the State-of-the-Art Video Theodolites—IATS in Monitoring Monitoring of artificial or natural structures, i.e., civil engineering structures or parts of the Earth’s surface, involves periodic or continuous observations to estimate the object’s general current state regarding its usability and stability, as well as a determination of the need for structure remediation, reconstruction, or destruction. The process involves performing different kinds of measurements using different sensors. The measurements and results must be precise and reliable, i.e., accurate, and tested for significance [8]. The results of the measurements represent an important parameter in assessing the condition and safety of the structures, and this is especially important for structures used beyond their designed lifetime. Any kind of damage or significant deformation affects the safety of the constructions, e.g., bridges, dams, towers, or skyscrapers, and this can result in their closure or even collapse. Monitoring of artificial or natural structures is one of the key tasks in engineering geodesy, next to site surveying and setting out. Geodetic monitoring is one aspect of monitoring systems in general. There are two subtypes of geodetic monitoring [8]: • Structural monitoring refers to the measurement and evaluation of civil engineer- ing structures such as bridges, tunnels, dams, railways, towers, or skyscrapers, i.e., generally manmade objects. • Geo-monitoring, in contrast, is used as a term for the determination of changes, movements, or deformation of natural structures, such as landslides and slopes. The main aim of geodetic monitoring is to determine statistically significant geometric changes in size, shape, and position between two or more measuring epochs. According to the monitoring data, action can be taken on the construction to prevent material and nonmaterial damages. Vibration-based monitoring, i.e., structural monitoring, has become common recently. Vibration-based monitoring consists of determining the dynamic dis- placements and natural frequencies of objects from different epochs of measurements. Any changes from the designed frequencies can be a sign of structural damage and a cause for alarm. Dynamic displacements and natural frequencies of objects can be determined by RTS and GNSS instruments, although their use in these projects has certain limitations. The limitation of the first RTS models was their instrument measurement frequency of 1 Hz, which was lower than the fundamental frequency of the bridge, as demonstrated in [65]. Possibilities of newer RTS models with measuring frequencies of 5–7 Hz were presented in [66–70], where RTS instruments were used for the measurement of simulated dynamic displacements to analyze the accuracy of dynamic measurements by RTS, as well as for the determination of dynamic displacements and natural frequencies of bridges in exploitation. The accuracy of an RTS instrument with 20 Hz measuring frequencies for recording changing 3D coordinates of a moving target was tested in [71] and for measuring dynamic displacements and natural frequencies of railway bridges in [72], where the RTS determined dynamic displacements of the bridge in vertical and lateral directions and the first five natural frequencies of the bridge. To overcome the limitations of RTS measuring frequencies, a new approach was shown in [6], where the authors demonstrated how to increase the measurement frequencies of an RTS instrument from 7–10 Hz to 20 Hz. A new approach was also demonstrated in a field experiment on a 74 m long footbridge for vertical bridge vibration measurements. Unlike RTS, GNSS instruments do not have limitations regarding measuring frequen- cies, since the newer GNSS instruments have a measuring frequency of up to 100 Hz. GNSS instruments are widely used for monitoring of dynamic displacements of large and flexible structures such as long-span bridges, towers, and high-rise buildings, which are charac- Appl. Sci. 2021, 11, 3893 20 of 29

terized by large displacements (10 cm or more) and lower natural frequencies (less than 1 Hz). Results of GNSS monitoring of dynamic displacements of large-scale structures were shown in [66,73–76], where dynamic displacements, i.e., natural frequencies, of structures were successfully determined from GNSS data. Determination of dynamic displacements of more rigid structures, which are characterized by smaller values of dynamic displace- ments, is limited due to the achievable measurement accuracy, which is on a centimeter level. Currently, many studies are focusing on applying GNSS instruments for monitoring the dynamic displacement of more rigid structures with dynamic displacements in the millimeter range and higher oscillation frequencies [67], as well as on improving the GNSS measurement accuracy due to the limitation of ephemeris, multipath, and atmospheric errors and receiver measurement noise [77]. The most commonly used geodetic instruments for structural and geo-monitoring include levels, RTS, and GNSS. Geodetic monitoring of engineering structures is usually done using a GNSS in combination with RTS. A GNSS offers great coverage, while an RTS offers high-precision measurements. The implementation of geodetic measurement sensors, i.e., instruments in complex construction monitoring systems, enables the so- called preventive maintenance of engineering structures and a transition from reactive maintenance, when damage has already occurred and possibly caused human casualties, to preventive maintenance [78]. In recent years, there has been a general trend toward the development of automated and autonomous monitoring. In addition to economics, which has constantly been the “problem”, another reason for the development of automated and autonomous measuring is the increased demand for continuous 24/7/365 monitoring systems. Densely built buildings and complex and construction demanding object projects require a continuous control of predefined thresholds. This also includes constructions in hazardous locations, which have not been used in the past. Likewise, today’s cities are expanding, and, due to a lack of land, tall buildings are being built, e.g., Burj Khalifa and similar skyscrapers all around the world. Precise 3D point measurement is either automated using an RTS with reflectors or using a GNSS. However, reflectors and GNSS cannot be used in every situation, because they cannot be placed on every point or object of interest. Furthermore, in the case of GNSS, a power supply must be ensured in sometimes hazardous and inaccessible places. These issues mean that we must use IATS with image sensors to collect spatial, spectral, and radiometrical information of the target point of interest and its environment, whereby, with the use of image analysis methods, RTS and GNSS methods can be replaced for determining precise 3D-point measurements of signalized points and unsignalized points [11]. Considering the aging of traffic infrastructure, a high number of bridges in particular are in need of rehabilitation around the world, which provides a huge opportunity for geodetic monitoring sensors (RTS, GNSS), especially IATS. Unfortunately, almost every 3 months, we are witnessing the collapse of bridges around the world. Any kind of damage or significant deformation affects the safety of the bridges, which can result in their closure or even collapse, thus disintegrating traffic systems [78]. In the next section, we give a few examples of IATS applications for the purpose of structural and geo-monitoring.

4.1. Applications in Structural and Geo-Monitoring One of the first applications of IATS for structural monitoring was conducted by the Chair of Geodesy, TU Munich, for the measurement of dynamic displacements and natural frequencies of the Fatih Sultan Mehmet Bridge over the Bosporus waterway in Istanbul, Turkey. Using a self-developed IATS prototype consisting of a Leica TPS1201 RTS with an ocular replaced by a 5 MPx CMOS color camera, they managed to determine vertical displacements of up to 50 cm and the first six natural frequencies of 0.106 Hz, 0.132 Hz, 0.176 Hz, 0.209 Hz, 0.272 Hz, and 0.333 Hz of the bridge during daily traffic. The determined frequencies coincided with the known frequencies from previous examination tests [7]. Appl. Sci. 2021, 11, 3893 21 of 29

Research conducted at the Institute of Engineering Geodesy and Measurement Sys- tems, TU Graz, covered a variety of experimental studies and in situ measurements using IATS Leica MS50 and MS60. They were aimed at determining the natural frequencies of the Augarten and Pongratz-Moore-Steg footbridges over the river Mur in Graz, Aus- tria. For reference measurements, they used three accelerometers which were oriented in the vertical direction, across and along the bridge. At Augarten bridge, using a Leica MS50 and instruments favorably placed along the longitudinal axis of the bridge [79], they managed to determine vertical displacements of up to 0.5 mm and a bridge frequency of 1.81 Hz using IATS (1.81 Hz using the accelerometer), which corresponded to a bridge natural frequency of 1.8 Hz [80]. At Pongratz-Moore-Steg bridge, using a Leica MS60 and instruments unfavorably placed along the transverse axis of the bridge [79], they managed to determine a bridge frequency of 2.5 Hz using IATS (2.5 Hz using the accelerometer) which corresponded to the step frequency of runners over the bridge. The displacements along the bridge axis (longitudinal and transverse) were too small to be resolved using the IATS placed in an unfavorable position, and they primarily depended on the reflectorless distance measurement accuracy of the IATS [79]. For geo-monitoring, the concept of using two IATS instruments for high-resolution, long-range stereo survey of georisk areas was investigated in the EU-FP7 project DE- MONTES, and the facts, objectives, results, and full report can be found in [81]. In the paper [82], the used methodology and a comparison of the main features with other terrestrial geodetic geo-monitoring methods were presented. The theoretically achievable accuracy of the measurement system was derived and verified using in situ data of a distant clay pit slope and simulated deformations. The deformations were simulated using movements of an artificial rock, a 1 m × 1 m structured and textured plate, which was fixed by two ropes and shifted and rotated to varying degrees during the different epochs. It was shown that the stereo IATS concept could obtain higher precision in the determination of 3D displacements, i.e., deformations, than other systems with comparable sensors. With the use of the telescope camera and telescope magnification, the images could be used for long-range applications. Dense point clouds with high single-point accuracy, including information about their precision, were generated. The 3D displacement vectors could be derived automatically and their significance specified. These results may be used to directly determine the rigid body motions of objects and surface patches. Compared to terrestrial laser scanner (TLS) measurements, long distances with smaller footprints are possible using the IATS approach, and there is no need for reference points within the region of interest. The large magnification of the telescope camera and the ability to capture image bundles further lead to a higher point density in comparison to simple photogrammetry approaches. The authors concluded that, by extrapolating the achievable accuracy to greater distances (by using a fixed base-to-distance ratio of 1:3 and the other input parameters of the field experiment), the following Helmert position errors were possible: for 1 km, 0.035 m; for 2 km, 0.069 m; for 3 km, 0.103 m [82].

4.2. Case Study In this section, we present our own experiment to contribute to the above-presented theoretical basis, along with conclusions reached regarding the potential benefits and possibilities of using IATS for structural monitoring. Because of the lack of a state-of-the-art IATS, as shown in Section 2.5, we developed our own IATS prototype consisting of a Leica TPS1201 RTS and a GoPro camera mounted on the ocular of the telescope (Figure 12, left). We designed an adapter for fitting the GoPro camera on the Leica TPS1201, made using a 3D printer. This offered the possibility to directly attach the camera to the ocular of the telescope of Leica TPS1201. Before conducting measurements, we performed a stability examination of the instrument’s telescope in the laboratory using a fitted camera in vertical direction. The examination showed that there were no movements of the telescope. The IATS prototype offered the possibility to manage the camera using the smartphone GoPro application, while we could manage the instrument using a laptop. Thus, we did not have Appl. Sci. 2021, 11, x FOR PEER REVIEW 22 of 29

Appl. Sci. 2021, 11, x FOR PEER REVIEW 22 of 29

examination of the instrument’s telescope in the laboratory using a fitted camera in verti- Appl. Sci. 2021, 11, 3893 22 of 29 calexamination direction. Theof the examination instrument’s showed telescope that in th theere laboratory were no movementsusing a fitted of camera the telescope. in verti- Thecal IATSdirection. prototype The examination offered the possibilityshowed that to thmanageere were the no camera movements using theof the smartphone telescope. GoProThe IATS application, prototype while offered we couldthe possibility manage theto manage instrument the camerausing a usinglaptop. the Thus, smartphone we did nottoGoPro touchhave application, theto touch instrument the while instrument or we the could camera, or managethe thereby camera, the ensuring instrumentthereby itsensuring stabilityusing aits laptop. along stability the Thus, horizontalalong we the did horizontalandnot verticalhave toand axes.touch vertical Before the instrumentaxes. conducting Before or conducti tests the incamera, theng field, tests thereby wein the performed ensuring field, we another its performed stability experiment along another the in experimentthehorizontal laboratory andin the where vertical laboratory we axes. tested whereBefore the abilitywe conducti tested of the ngthe IATStests ability prototypein theof the field, IATS to we determine prototype performed predefined to another deter- mineamplitudesexperiment predefined within the amplitudes different laboratory frequencies with where different we using tested frequencies a multi-purposethe ability using of the a universal multi-purposeIATS prototype testing universal machineto deter- testingintendedmine predefined machine for static intended amplitudes and dynamic for static with testingand different dynamic of mechanicalfrequencies testing of propertiesusing mechanical a multi-purpose of properties building materialsofuniversal build- ingandtesting materials constructions. machine and intended constructions. The test for consisted static The and test of dynamic simulatedconsisted testing of amplitudes simulated of mechanical ofamplitudes±0.2, properties±0.5, of± ±0.2,1.0, of build-±±0.5,2.0, ±1.0,anding materials±±2.0,5.0 mmand and±5.0 at frequencies constructions.mm at frequencies of 0.2, The 0.5, testof 1.0, 0.2, cons 2.0, 0.5,isted and 1.0, 5.0of 2.0, simulated Hz. and We 5.0 managed Hz.amplitudes We managed to determine of ±0.2, to ±0.5,de- all termineamplitudes±1.0, ±2.0, all andamplitudes and ±5.0 all frequencies,mm and at frequenciesall frequencies, as partially of 0.2, as presented partially0.5, 1.0, 2.0, inpresented [ 83and]. 5.0 inHz. [83]. We managed to de- termineTheThe conductedall conducted amplitudes tests and in the theall fieldfrequencies,field werewere performed perf as ormedpartially on on thepresented the railway railway in bridge [83].bridge Kloštar Kloštar after after its itsreconstruction reconstructionThe conducted for for the teststhe purpose purpose in the of field loadof load were testing testing perf (Figureormed (Figure 12 on, 12, middle). the middle). railway Since Since bridge the the bridge Kloštar bridge did afterdid not nothaveits reconstructionhave any any characteristic characteristic for the points, purpose points, i.e., i.e.,of unsignalized load unsi testinggnalized targets (Figure targets suitable 12, suitable middle). for imagefor Since image processing the processing bridge and did andtargetnot targethave identification any identification characteristic between between points, frames, frames, i.e., we unsi used wegnalized used predefined predefined targets photo suitable photo marks for marks (Figureimage (Figure processing 12, right) 12, right)forand target target for target detection identification detection between between between frames. frames. frames, The The photowe photo used marks marks predefined were were four four photo different different marks circlescircles (Figure with 12, predefined diameters (5 mm, 10 mm, 15 mm, and 20 mm), as well as a predefined horizontal predefinedright) for target diameters detection (5 mm, between 10 mm, frames. 15 mm, The and photo 20 mm), marks as well were as four a predefined different circleshorizontal with spacing between the circle centers of 40 mm and a vertical spacing of 20 mm, as shown in spacingpredefined between diameters the circle (5 mm, centers 10 mm, of 4015 mmmm, an andd a20 vertical mm), as spacing well as of a predefined20 mm, as shownhorizontal in Figure 12 (right). The IATS prototype was placed at a distance of 28.519 m perpendicular Figurespacing 12 between (right). The the IATScircle prototype centers of was 40 mm placed and at a avertical distance spacing of 28.519 of 20m perpendicularmm, as shown to in to the longitudinal axis of the bridge in the middle of the bridge span (Figure 13). theFigure longitudinal 12 (right). axis The of IATS the bridge prototype in the was middle placed of atthe a distancebridge span of 28.519 (Figure m 13).perpendicular to the longitudinal axis of the bridge in the middle of the bridge span (Figure 13).

FigureFigure 12. 12. IATSIATS prototype prototype ( (leftleft),), railway railway bridge bridge Klošta Kloštarr over over the the river river Dobra Dobra ( (middle),), and and photo photo mark mark ( right). Figure 12. IATS prototype (left), railway bridge Kloštar over the river Dobra (middle), and photo mark (right).

Figure 13. IATS prototype setup in the field in relation to the bridge. FigureFigure 13.13. IATSIATS prototypeprototype setupsetup inin thethe fieldfield in in relation relation to to the the bridge. bridge. This way, we achieved the optimal position of the IATS prototype relative to the bridgeThisThis for determination way,way, wewe achievedachieved of the thethe bridge’s optimaloptimal vertical positionposition displacements, ofof thethe IATSIATS since prototypeprototype the longitudinal relativerelative toto axis thethe ofbridgebridge the bridge forfor determinationdetermination was parallel ofof to thethe the bridge’sbridge’s image verticalverticalsensor. displacements,displacements,Using the instrument’s sincesince thethe longitudinaltelescopelongitudinal at axis30×axis magnification,ofof thethe bridgebridge waswasthe parallelcameraparallel torecordedto thethe image image a movie sensor. sensor. with Using Using the the followingthe instrument’s instrument’s settings: telescope telescope narrow at FOV,at 30 30×× magnification,magnification, thethe cameracamera recordedrecorded aa moviemovie withwith thethe followingfollowing settings:settings: narrow FOV, 1920 × 1080 px, ay 60 fps. With this setup, we achieved the smallest circle with 5 mm diameter and 15 pixels, i.e., one pixel corresponded to 0.333 mm for this case study setup in the field. We also used accelerometers for measuring the frequency response of the bridge. Appl. Sci. 2021, 11, x FOR PEER REVIEW 23 of 29

Appl. Sci. 2021, 11, x FOR PEER REVIEW 23 of 29

1920 × 1080 px, ay 60 fps. With this setup, we achieved the smallest circle with 5 mm di- Appl. Sci. 2021, 11,1920 3893ameter × 1080 and px, 15 ay pixels, 60 fps. i.e., With one this pixel setup, corresponded we achieved to 0.333 the smallest mm for thiscircle case with study 5 mm setup di- in 23 of 29 ameterthe field. and 15We pixels, also used i.e., accelerometersone pixel corresponded for measuring to 0.333 the mm frequency for this response case study of setupthe bridge. in theTrains field. Wepassed also along used accelerometersthe bridge at different for measuring speeds. the In thisfrequency study, responsewe show ofthe the results bridge. from Trainstwo passedtests, i.e., along when the the bridge trains at passeddifferent at speed40 km/hs. In and this 60 study, km/h. we show the results from Trains passed along the bridge at different speeds. In this study, we show the results from two tests,For i.e., processing when the the trains recorded passed movie at 40 km/hfrom andthe camera,60 km/h. we developed an algorithm in two tests, i.e., when the trains passed at 40 km/h and 60 km/h. MATLAB.For processing The algorithm the recorded performed movie several from the steps. camera, First, we the developed movie format an algorithm was converted in For processing the recorded movie from the camera, we developed an algorithm in MATLAB.into frames, The wherebyMATLAB.algorithm each performed The image algorithm represente several performed steps.d one First, several measuring the steps.movie epoch. First, format Then, the was movie the converted image format was was converted intotransformed frames, whereby intointo frames,am each orthogonal image whereby represente projection each imaged one according represented measuring to the oneepoch. known measuring Then, azimuth the epoch. image and Then, waszenith the image was transformedangle of the into transformedIATS am prototypeorthogonal into to amprojection the orthogonal longitudinal according projection axis to of the the according known bridge, azimuth toi.e., the to knownthe and position zenith azimuth of and zenith anglethe ofphoto the IATSmark.angle prototype Subsequently, of the IATS to the prototypeto longitudinal reduce tocomp the axisutation longitudinal of the time, bridge, the axis regioni.e., of theto aroundthe bridge, position the i.e., photo toof the position of themark photo was mark. defined.the Subsequently, photo Then, mark. a conversion to Subsequently, reduce comp from toutation an reduce RGB time, computationimage the to region a binary time, around black-and-white the the region photo around the photo markimage was was defined. performed,mark Then, was defined.afacilitating conversion Then, the from adetection conversion an RGB of image circle from antoboundaries a RGB binary image black-and-whiteand to circle a binary centers. black-and-white imageAccording was performed, toimage the pixel was facilitating coordinates performed, the of facilitatingdetection circle cent ofers, the circle the detection boundariesjoint center of circle ofand all boundaries circle four circlescenters. and was circle centers. Accordingcalculated. to theTheAccording pixel differences coordinates to the between pixel of circle coordinatesjoint centerscenters, of of the circle each joint image centers, center for theof every all joint four test center circles in relation of was all four to circles was calculated.the first referenceThe calculated.differences imageThe betweenbefore differences the joint train centers betweenpassed of could each joint centersimagethen be for ofcalculated. every each image test Accordingly, in for relation every to test we in relation to thecould first reference calculatethe imagethe first dynamic reference before thevertical image train displacemebeforepassed the could trainnts then in passed pixels. be calculated. could As a thenfunction Accordingly, be calculated. of the known we Accordingly, we couldspacing calculate between couldthe dynamic the calculate circle vertical centers, the dynamic displaceme the scale vertical ntsfactor in displacements pixels. was calculated As a function in pixels. and ofused As the a to functionknown convert of the known spacingpixel betweendisplacementsspacing the circle betweeninto centers,millimeter the circlethe displacescale centers, factorments. the was scale Lastly, calculated factor using was and calculated fast used Fourier to and convert transfor- usedto convert pixel pixelmation, displacements the timedisplacements domain into millimeter was into converted millimeter displace into displacements.ments. the frequency Lastly, Lastly,using domain, usingfast Fourierand fast the Fourier transfor-natural transformation, fre- the mation,quencies the oftime thetime domain bridge domain werewas was converteddetermined. converted into into the the frequency frequency domain, domain, and and the the natural natural fre- frequencies of the quenciesFigure of the 14 bridgebridge (left) were wereshows determined.determined. the computed first modal shape of the bridge at 5.35 Hz fre- quency,Figure and 14 (left)Figure Figureshows 15 (left) 14the (left)shows computed shows the second first the computedmo modaldal shape shape first of at modal the 7.77 bridge Hz shape frequency at of5.35 the Hz using bridge fre- the at 5.35 Hz fre- quency,finite andelement Figurequency, method 15 (left) and (FEM). Figureshows In the15 Figures (left) second shows 14modal and the shape15 second (right), at modal7.77 the Hz shapefirst frequency and at 7.77second using Hz frequencymodal the using the finiteshapes element and frequenciesmethodfinite element (FEM). obtained method In Figures from (FEM). ambient 14 In and Figures vibration15 (right), 14 and measurements the15 (right), first and the secondusing first and accelerom- modal second modal shapes shapeseters and are frequenciesshown.and frequenciesDifferences obtained obtainedbetween from ambient fromthe FEM ambient vibration and ambient vibration measurements vibration measurements using measurements accelerom- using accelerometers oc- are eterscurred are shown. regularlyshown. Differences because Differences the between FEM betweenmodel the FEM was the and made FEM ambient using and ambientdesignvibration parameters vibration measurements which measurements oc-could occurred curredhave regularly differedregularly frombecause the becausetheconstructed FEM themodel FEMbridge. was model made was using made design using parameters design parameters which could which could have have differed fromdiffered the constructed from the constructed bridge. bridge.

5.35 Hz 5.49 Hz 5.35 Hz 5.49 Hz Figure 14. First modal shape and frequency computed by FEM (left) and determined by accel- FigureFigure 14.erometersFirst 14. modalFirst (right modal shape). shape and frequency and frequency computed computed by FEM by (FEMleft) and(left determined) and determined by accelerometers by accel- (right). erometers (right).

7.77 Hz 6.85 Hz 7.77 Hz 6.85 Hz Figure 15. Second modal shape and frequency computed by FEM (left) and determined by accelerometers (right). Figure 15. Second modal shape and frequency computed by FEM (left) and determined by accel- erometers (right). Figure 15. Second modalThe shape aim and of the frequency testing computed was to determine by FEM (left the) naturaland determined frequencies by accel- of the bridge during erometers (right). vibration monitoring using the IATS prototype. To assess the accuracy of our IATS pro- totype, we used ambient vibration measurements by accelerometers as a reference. The first natural frequency determined by the IATS prototype measurements matched the first natural frequency of the bridge measured by accelerometers, while the second natural frequency determined by IATS slightly differed from that determined by accelerometers Appl. Sci. 2021, 11, x FOR PEER REVIEW 24 of 29 Appl. Sci. 2021, 11, x FOR PEER REVIEW 24 of 29

The aim of the testing was to determine the natural frequencies of the bridge during The aim of the testing was to determine the natural frequencies of the bridge during vibration monitoring using the IATS prototype. To assess the accuracy of our IATS proto- vibration monitoring using the IATS prototype. To assess the accuracy of our IATS proto- type, we used ambient vibration measurements by accelerometers as a reference. The first Appl. Sci. 2021, 11, 3893type, we used ambient vibration measurements by accelerometers as a reference. The first 24 of 29 natural frequency determined by the IATS prototype measurements matched the first nat- natural frequency determined by the IATS prototype measurements matched the first nat- ural frequency of the bridge measured by accelerometers, while the second natural fre- ural frequency of the bridge measured by accelerometers, while the second natural fre- quency determined by IATS slightly differed from that determined by accelerometers for quency determined by IATS slightly differed from that determined by accelerometers for both tests (Figuresfor both 16 and tests 17). (Figures At 40 km/h,16 and the 17 ).first At frequency, 40 km/h, the F1 = first 5.49 frequency, Hz, detected F1 = by 5.49 the Hz, detected by both tests (Figures 16 and 17). At 40 km/h, the first frequency, F1 = 5.49 Hz, detected by the accelerometer thewas accelerometer equally detected was by equally the IATS detected prototype, by the F IATS1 = 5.49 prototype, Hz. At 40 F1 km/h,= 5.49 the Hz. At 40 km/h, accelerometer was equally detected by the IATS prototype, F1 = 5.49 Hz. At 40 km/h, the second frequency,the second F2 = 6.85 frequency, Hz, detected F2 = 6.85 by the Hz, accelerometer detected by the was accelerometer detected by was the detected IATS by the IATS second frequency, F2 = 6.85 Hz, detected by the accelerometer was detected by the IATS prototype as Fprototype2 = 6.78 Hz. as At F2 60= 6.78 km/h, Hz. the At first 60 km/h, frequency, the first F1 = frequency, 5.49 Hz, detected F1 = 5.49 by Hz, the detected by the prototype as F2 = 6.78 Hz. At 60 km/h, the first frequency, F1 = 5.49 Hz, detected by the accelerometer accelerometerwas equally detect was equallyed by the detected IATS prototype, by the IATS F1 prototype,= 5.49 Hz. At F1 =60 5.49 km/h, Hz. the At 60 km/h, the accelerometer was equally detected by the IATS prototype, F1 = 5.49 Hz. At 60 km/h, the second frequency,second F2 = frequency, 6.85 Hz, detected F2 = 6.85 by Hz, the detected accelerometer by the accelerometerwas detected by was the detected IATS by the IATS second frequency, F2 = 6.85 Hz, detected by the accelerometer was detected by the IATS prototype as Fprototype2 = 6.82 Hz. as The F2 = conducted 6.82 Hz. The study conducted showed studythat IATS showed can thatbe used IATS for can the be used for the prototype as F2 = 6.82 Hz. The conducted study showed that IATS can be used for the determination determinationof dominant vertical of dominant natural vertical frequencies, natural confirming frequencies, the confirming previous studies the previous studies determination of dominant vertical natural frequencies, confirming the previous studies performed by performed[7,79,80]. In by our [7 ,future79,80]. work, In our we future will work,present we all will conducted present tests all conducted performed tests performed performed by [7,79,80]. In our future work, we will present all conducted tests performed during this loadduring testing. this load testing. during this load testing.

Figure 16. AmplitudeFigure 16. Amplitude spectrum spectrum of vertical of component vertical component and detected and detected first and first second and second frequencies frequencies by IATS prototype Figure 16. Amplitude spectrum of vertical component and detected first and second frequencies by IATS prototype (excitation by train passing at 40 km/h). (excitation byby trainIATS passing prototype at 40 (excitation km/h). by train passing at 40 km/h).

Figure 17. Amplitude spectrum of vertical component and detected first and second frequencies Figure 17. Amplitude spectrum of vertical component and detected first and second frequencies Figure 17. Amplitudeby IATS prototype spectrum (excitation of vertical by train component passing and at 60 detected km/h). first and second frequencies by IATS prototype (excitation byby trainIATS passing prototype at 60 (excitation km/h). by train passing at 60 km/h). 5. Discussion 5. Discussion The conducted5. Discussion research regarding the technological development of photo and video The conducted research regarding the technological development of photo and video theodolites showed that their development was slow and largely dependent on the devel- theodolites showedThe that conducted their development research regardingwas slow and the technologicallargely dependent development on the devel- of photo and video opment of distance-meters, image sensors, motorization, and their integration into the opment of distance-meters,theodolites showed image thatsensors, their motorization, development and was their slow integration and largely into dependent the on the de- theodolites. Initially, a theodolite supported a photo camera for the purpose of determin- theodolites. Initially,velopment a theodolite of distance-meters, supported a image photo sensors, camera motorization,for the purpose and of theirdetermin- integration into the ing the elements of the exterior orientation. These so-called photo theodolites were used ing the elementstheodolites. of the exterior Initially, orientation. a theodolite These supported so-called a photo photo camera theodolites for the were purpose used of determining the elements of the exterior orientation. These so-called photo theodolites were used for many mapping projects because surveying was faster and easier, while they also offered the possibility of surveying larger areas with satisfying accuracy. With the development of EDM and the principle of the difference in two currents in a photoelectric cell, static targets could be recognized and measured with high accuracy. Appl. Sci. 2021, 11, 3893 25 of 29

The developed image sensors were then integrated into the theodolites and their purpose changed from mapping to object tracking. The demands of the automotive and aerospace industries regarding assembly and quality control in the 1980s forced companies to build electronic tachymeters with integrated image sensors into servomotorized theodolites. The greatest contribution from this development line was that image sensors were used for autofocusing and for determining the target position relative to the optical axis of the telescope. Although the technological development of commercial instruments slowed down in the 1990s, the academic community maintained research efforts and developed the applications of digital image processing methods and computer vision, which have been preserved to this day. Initial research focused on detecting artificial targets and accurate aiming with the help of cameras for the purpose of aiming and measuring unsignalized tar- gets. Solutions based on artificial intelligence were developed. By implementing the expert knowledge of sighting points into software solutions, artificial intelligence was applied, and expert systems were developed. The developed algorithms were later implemented in commercial instruments. By automating the measurements, the level of achieved accuracy was increased. The year 2005 saw the introduction of IATS that we know today. Technological de- velopments regarding the different sensors, i.e., hardware and software, enabled their integration into the instruments. Following their fusion, IATS have been more frequently used for the tasks of structural and geo-monitoring. The currently used sensors for struc- tural monitoring and for determining a structure’s vibrations and dynamic displacements are accelerometers, GNSS, and RTS. These sensors present some drawbacks, whereby access to the monitored structure necessitates them being installed on the structure with a power supply being ensured. Using accelerometers, it is only possible to evaluate the vibrations of the structures [64]. Furthermore, the dynamic measurement accuracy of GNSS instru- ments (in kinematic measurement mode) is on a centimeter level with a high-frequency measurement sampling rate (up to 100 Hz); in contrast, RTS can measure displacement on a millimeter level but with a lower-frequency measurement sampling rate (5–20 Hz). It is obvious that these two measurement techniques do not provide an optimal solution for structural monitoring. According to the conducted research studies shown in Section4, it is evident that IATS have overcome the major shortcomings of RTS and GNSS instruments in the projects of structural and geo-monitoring. Access to the monitored structure is not required when using IATS, which allows a more flexible definition of the measurement points. Moreover, the achievable accuracy and precision are also on a higher level regarding the RTS and GNSS. The briefly presented case study in the article shows that the abovementioned draw- backs of accelerometers, GNSS, and RTS can be overcome by using IATS (an IATS prototype was sued in our case study). The developed IATS prototype highlighted its major advan- tages and potential for the purpose of structural monitoring in contrast to other sensors. The precision and accuracy of measured displacements were on a high level since image- based measurements were obtained using a GoPro camera at the 30× optical magnification of TS Leica TPS1201. Using recordings at 30 fps or 60 fps, the determination of high natural frequencies of the structures was possible. The prototype does not need to be placed on the structure; hence, access to the monitored structure is not necessary. In addition to natural frequencies, we can determine the static and dynamic displacements in the vertical and horizontal planes (longitudinal and transverse depending on the position of the prototype relative to the structure). Future work is needed in this direction since we have only performed studies for the determination of vertical displacements and natural frequencies, while studies regarding displacements along the longitudinal or transverse axis are planned. As shown in Section4, state-of-the-art IATS can perform all these tasks; however, the price of the presented IATS prototype is much lower than that of a commercial IATS, which is also an important parameter to consider. Using the proposed image-based measurements, the detection of object movements perpendicular to the IATS line of sight is possible. To measure movements along the Appl. Sci. 2021, 11, 3893 26 of 29

sighting axis, the IATS EDM can be used; alternatively, if available, IATS scanner data can be used, i.e., fusion of laser scans and camera data to RGB + D images. This approach combines the advantages of the two methods. This is also possible with the latest generations of IATS, i.e., IASTS, as explained in Section 3.3. At the moment, a few studies exist on this topic; however, further work is needed, because the possibilities and advantages offered by this approach for engineering tasks in geo-monitoring are extensive, especially in hazardous and difficult-to-access areas, necessitating thorough testing and analysis. With the integration of cameras into total stations, so-called IATS, these different sensor classes have been fused into one single universal instrument [5]. This integration offers a wide coverage of different geodetic tasks to be resolved much more quickly, easily, and precisely in comparison with classical geodetic methods and instruments. Furthermore, it offers possibilities to solve some problems and perform some tasks which were impossible in the past using classic TS, RTS, and GNSS.

Author Contributions: Conceptualization, R.P.; methodology, R.P.; validation, R.P., M.R., and A.M.; investigation, R.P., M.R., A.M., and S.M.; resources, R.P. and A.M.; writing—original draft preparation, R.P.; writing—review and editing, M.R. and A.M.; visualization, R.P. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: The data presented in this study are available on request from the corresponding author. The data are not publicly available [they are part of the official load test of the Kloštar bridge, and as such are not public available]. Conflicts of Interest: The authors declare no conflict of interest.

References 1. Bannister, A.; Raymond, S.; Baker, R. Surveying, 6th ed.; Longman Scientific & Technical: London, UK, 1992; ISBN 0-470-21845-2. 2. Benˇci´c,D.; Solari´c,N. Mjerni Instrumenti i Sustavi u Geodeziji i Geoinfromatici; Školska Knjiga: Zagreb, Croatia, 2008. 3. Ogundare, J.O. Precise Surveying; John Wiley & Sons: Hoboken, NJ, USA, 2016; ISBN 978-1-119-10251-9. 4. Lemmens, M. Total Stations: The Surveyor’s Workhorse. GIM Int. 2016, 30, 20–25. 5. Wunderlich, T.; Wasmeier, P.; Wagner, A. Auf dem Weg zum geodätischen Universalinstrument—Wie nahe am Ziel sind IATS und MS50? In Proceedings of the Terrestrisches Laserscanning 2014 (TLS 2014), Fulda, Germany, 11–12 December 2014; pp. 177–192. 6. Lienhart, W.; Ehrhart, M.; Grick, M. High frequent total station measurements for the monitoring of bridge vibrations. J. Appl. Geod. 2017, 11, 1–8. [CrossRef] 7. Wagner, A.; Wasmeier, P.; Reith, C.; Wunderlich, T. Bridge Monitoring by Means of Video-Tacheometer—A Case Study. AVN Allg. Vermess. Nachr. 2013, 120, 283–292. 8. Wagner, A. New Geodetic Monitoring Approaches Using Image Assisted Total Stations. Ph.D. Thesis, Technische Universität München, München, Germany, 2017. 9. Koppe, C. Die Photogrammetrie Oder Bildmesskunst; Verlag der Deutschen Photographen-Zeitung: Weimar, Germany, 1889. 10. Ott, M. Photogrammetrischer Theodolit für Hochgebirgsaufnahmen: Nach Angabe von Professor Dr. S. Finsterwalder in München neu konstruiert. Z. Instrum. 1895, 15, 370–373. 11. Walser, B.H. Development and Calibration of an Image Assisted Total Station. Ph.D. Thesis, ETH Zürich, Zürich, Switzerland, 2004. 12. Alexander, R.; Lehmann, M.; Kahmen, H.; Paar, G.; Miljanovic, M.; Ali, H.; Egly, U.; Eiter, T. A 3D optical deformation measurement system supported by knowledge-based and learning techniques. J. Appl. Geod. 2009, 3, 1–13. [CrossRef] 13. Knoblach, S. Entwicklung, Kalibrierung und Erprobung eines Kameraunterstützten Hängetachymeters. Ph.D. Thesis, Technische Universität Dresden, Dresden, Germany, 2009. 14. Wasmeier, P. Grundlagen der Deformationsbestimmung mit Messdaten Bildgebender Tachymeter. Ph.D. Thesis, Technische Universität München, München, Germany, 2009. 15. Reiterer, A.; Wagner, A. System Considerations of an Image Assisted Total Station—Evaluation and Assessment. AVN Allg. Vermess.-Nachr. 2012, 119, 83–94. 16. Huep, W. Scannen mit der trimble VX spatial station. Z. Vermess. 2010, 135, 330–336. 17. Gruner, H. Photogrammetry: 1776–1976. Photogramm. Eng. Remote Sens. 1977, 43, 569–574. Appl. Sci. 2021, 11, 3893 27 of 29

18. Frankich, K. Close Range Photogrammetric Systems and Their Applications in Ophthalmology. Ph.D. Thesis, The University of British Columbia, Vancouver, BC, Canada, 1973. 19. Finsterwalder, R.; Hofmann, W. Photogrammetrie, 3rd ed.; Walter de Gruyter & Co.: Berlin, Germany, 1968. 20. Leica Geosystems 200 Years of Surveying History. Available online: https://leica-geosystems.com/about-us/summary/history (accessed on 25 March 2021). 21. Gigas, E.; Ebeling, K. Elektrisches Auge. In DGK; Institut für Angewandte Geodäsie: Frankfurt, Germany, 1957; Volume B. 22. Wagner, A.; Wasmeier, P.; Wunderlich, T.; Ingensand, H. Vom selbstzielenden Theodolit zur Image Assisted Total Station From the Self-aiming Theodolite to Image Assisted Total Stations. AVN Allg. Vermess. Nachr. 2014, 121, 171–180. 23. C & D Professional Services Contraves EOTS Cinetheodolite. Available online: http://www.cdproservices.com/product_0903g. html (accessed on 25 March 2021). 24. Hilster, N. de 1980 SAT AGA-Minilir. Available online: http://www.dehilster.info/geodetic_instruments/1980_sat-sagem_AGA- Minilir.php#note1 (accessed on 25 March 2021). 25. Wentzell, H.F. Polarfix: A new concept in high accuracy position fixing. World Dredg. Mar. Constr. 1983, 19, 9. 26. Ingensand, H. 2000 Jahre Theodolit: Von Heron bis Heinrich Wild. Vermess. Photogramm. Kult. 1992, 90, 128–140. 27. Kahmen, H.; Suhre, H. Ein lernfähiges tachymetrisches Vermessungssystem zur Überwachung kinematischer Vorgänge ohne Beobachter. Z. Geodäsie Geoinf. Landmanag. 1983, 108, 345–351. 28. Kahmen, H.; Steudel, J. Das automatisch zielende Messsystem GEOROBOT II. In Proceedings of the Xth Internationalen Kurs für Ingenieurvermessung, München, Germany, 12–17 September 1988. 29. Gottwald, R. Kern E2-SE—Ein neues Instrument nicht nur für die Industrievermessung? AVN Allg. Vermess. Nachr. 1987, 147–153. 30. Katowski, O. Deformationsmessung an Bauwerken mit dem automatischen TheodolitmessSystem ATMS. In Optical 3-D Measure- ment Techniques; Wichmann: Karlsruhe, Germany, 1989; pp. 393–403. 31. Kennie, T.J.M.; Petrie, G. (Eds.) Engineering Surveying Technology; Taylor & Francis Group: London, UK; New York, NY, USA, 1990; Volume 27, ISBN 0751401935. 32. Lienhart, W. Integrated Digital Imaging and Its Impact on Survey Workflows; Visualisation Technical; PositionIT: Muldersdrift, South Africa, 2011; pp. 51–55. 33. Kahmen, H.; Roi´c,M. A new generation of measurement robots for object reconstruction without targeting. In Proceedings of the Optical 3d Measurements III, Vienna, Austria, 2–4 October 1995; pp. 251–262. 34. Fabiankowitsch, J. Automatische Richtungsmessung mit Digitalen Differenzbildern; TU Vienna: Vienna, Austria, 1990. 35. Mischke, A. Entwicklung Eines Videotheodolit-Meßsystems zur Automatischen Richtungsmessung von Nicht Signalisierten Objektpunkten; TU Vienna: Vienna, Austria, 2000; Volume 54. 36. Reiterer, A.; Vicovac, T.; Egly, U.; Mai, E.; Moafipoor, S.; Grejner-Brzezinska, D.A.; Toth, C.K. Application of artificial intelligence in Geodesy—A review of theoretical foundations and practical examples. J. Appl. Geod. 2010, 4, 201–217. [CrossRef] 37. Roi´c,M. Erfassung von Nicht Signalisierten 3D-Strukturen mit Videotheodoliten. Ph.D. Thesis, Technische Universität Wien, Vienna, Austria, 1994. 38. Matthias, H. Der Roboter-Theodolit Topomat: Technik, Anwendungen und Auswirkungen auf den Beruf. Vermess. Photogramm. Kult. 1991, 8, 427–431. [CrossRef] 39. Topcon Topcon Museum|Total Station. Available online: https://www.topcon.co.jp/en/positioning/museum/ts/ (accessed on 15 March 2021). 40. Ingensand, H.; Boeckem, B. Automatic Location and Pointing Techniques in Local Positioning Systems. In Proceedings of the 4th Conference on Optical 3-D Measurement Techniques, Zürich, Switzerland, 29 September–2 October 1997; pp. 329–338. 41. Topcon GPT-7000i Specifications. Available online: http://www.topcon.si/zares4/dokumenti/gpt-7000i.pdf (accessed on 25 March 2021). 42. Topcon GPT-9000Ai Specifications. Available online: https://totalstations.co/wp-content/uploads/2017/07/Topcon-GPT-9001 A-Edited-1.pdf (accessed on 11 March 2021). 43. Trimble Trimble VX Spatial Station. Available online: http://trl.trimble.com/docushare/dsweb/Get/Document-348124/022543- 261F_TrimbleVX_DS_0110_lr.pdf (accessed on 25 March 2021). 44. Pentax R-400VDN Series. Available online: https://pdf.archiexpo.com/pdf/pentax-precision/r-400vdn-series/151417-284341. html (accessed on 25 March 2021). 45. Leica Geosystems, A. Leica Viva TS11 Datasheet. Available online: https://leica-geosystems.com/-/media/files/ leicageosystems/products/brochures/tender-flexline-plus/leica-viva-ts11-ds-lr.ashx?la=en&hash=8467ACC4E2E31C6 DFF7C1497091D3692 (accessed on 25 March 2021). 46. Leica Geosystems, A. Leica Viva TS15 Datasheet. Available online: http://kumonos.info/downloads/datasheet_ts15.pdf (accessed on 25 March 2021). 47. Wagner, A.; Wiedemann, W.; Wasmeier, P.; Wunderlich, T. Improved concepts of using natural targets for geo-monitoring. In Proceedings of the 3rd Joint International Symposium on Deformation Monitoring (JISDM), Vienna, Austria, 30 March–1 April 2016. 48. Wagner, A.; Wiedemann, W.; Wunderlich, T. Fusion of Laser Scan and Image Data for Deformation Monitoring—Concept and Perspective. In Proceedings of the INGEO—7th International Conference on Engineering Surveying, Lisbon, Portugal, 18–20 October 2017; pp. 157–164. Appl. Sci. 2021, 11, 3893 28 of 29

49. Whitt, M. Trimble S Series Total Station Comparison—Positioning Solutions. Available online: https://positioningsolutions.com/ blogs/product-info/total-station-comparison (accessed on 25 March 2021). 50. Topocn DS-200i Series Imaging Robotic Total Station. Available online: https://www.topconpositioning.com/sites/default/files/ product_files/ds-200i_series_broch_7010_2160_revc_sm.pdf (accessed on 25 March 2021). 51. Maar, H.; Zogg, H.-M. Leica Nova MS60 White Paper. Available online: http://leica-geosystems.com/products/total-stations/ multistation/leica-nova-ms60 (accessed on 25 March 2021). 52. Trimble Trimble SX10|Scanning Total Station. Available online: https://geospatial.trimble.com/products-and-solutions/trimble- sx10 (accessed on 25 March 2021). 53. Trimble Trimble SX12|Scanning Total Station. Available online: https://geospatial.trimble.com/SX12 (accessed on 25 March 2021). 54. Topcon Topcon GTL-1000 Laser Scanner Total Station. Available online: https://www.topcon.co.jp/en/positioning/products/ pdf/GTL-1000_E.pdf (accessed on 25 March 2021). 55. Wagner, A. A new approach for geo-monitoring using modern total stations and RGB + D images. Meas. J. Int. Meas. Confed. 2016, 82, 64–74. [CrossRef] 56. Huep, W. Konzeption und Einsatzmöglichkeiten eines automatisierten Theodolitsystems. In Proceedings of the Xth Internationaler Kurs für Ingenieurvermessung, München, Germany, 12–17 September 1988. 57. Grimm, D.E.; Zogg, H.-M. Leica Nova MS50 White Paper. Available online: https://www.google.hr/url?sa=t&rct=j&q=&esrc=s& source=web&cd=&cad=rja&uact=8&ved=2ahUKEwiVlMOpxKDvAhVmyoUKHeveD-IQFjAAegQIAhAD&url=https%3A% 2F%2Fw3.leica-geosystems.com%2Fdownloads123%2Fzz%2Ftps%2Fnova_ms50%2Fwhite-tech-paper%2Fleica_nova_ms50_ tpa_en.pdf& (accessed on 25 March 2021). 58. Grimm, D.; Kleemaier, G.; Zogg, H.-M. ATRplus White Paper. Available online: https://leica-geosystems.com/-/media/files/ leicageosystems/products/white-papers/atrplus_wp.ashx?la=en&hash=91E09F6A1CF79DE10D9B742391F7B35F (accessed on 25 March 2021). 59. Ehrhart, M.; Lienhart, W. Development and evaluation of a long range image-based monitoring system for civil engineering structures. In Proceedings of the SPIE 9437, Structural Health Monitoring and Inspection of Advanced Materials, Aerospace, and Civil Infrastructure 2015, Aerospace, San Diego, CA, USA, 9–12 March 2015. 60. Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision; Cambridge University Press: Cambridge, UK, 2004; ISBN 9780521540513. 61. Vogel, M. Vom Pixel zur Richtung Die Räumlichen Beziehungen Zwischen Abbildungsstrahlen und Tachymeter-Richtungen. Ph.D. Thesis, Technische Universität Darmstadt, Darmstadt, Germany, 2006. 62. Wagner, A.; Wiedemann, W.; Wunderlich, T. Using IATS to Read and Analyze Digital Leveling Staffs. In Proceedings of the International Symposium on Engineering Geodesy—SIG 2016, Varaždin, Croatia, 20–22 May 2016; pp. 515–526. 63. Wagner, A.; Wiedemann, W.; Wasmeier, P.; Wunderlich, T. Monitoring Concepts using Image Assisted Total Stations. In Proceedings of the International Symposium on Engineering Geodesy—SIG 2016, Varaždin, Croatia, 20–22 May 2016; pp. 137–148. 64. Ehrhart, M.; Lienhart, W. Monitoring of Civil Engineering Structures using a State-of-the-art Image Assisted Total Station. J. Appl. Geod. 2015, 9.[CrossRef] 65. Cosser, E.; Roberts, G.W.; Meng, X.; Dodson, A. Measuring the dynamic deformation of bridges using a total station. In Proceedings of the 11th FIG Symposium on Deformation Measurements, Santorini, Greece, 25–28 May 2003; pp. 605–612. 66. Lekidis, V.; Tsakiri, M.; Makra, K.; Karakostas, C.; Klimis, N.; Sous, I. Evaluation of dynamic response and local soil effects of the Evripos cable-stayed bridge using multi-sensor monitoring systems. Eng. Geol. 2005, 79, 43–59. [CrossRef] 67. Psimoulis, P.; Moschas, F.; Stiros, S. Measuring the Displacements of a rigid footbridge using geodetic instruments and an ac-celerometer. In Proceedings of the 1st Joint International Symposium on Deformation Monitoring, Hong Kong, China, 2–4 November 2011. 68. Gikas, V.; Daskalakis, S. Comparative testing and analysis of RTS vs. In GPS for structural monitoring using calibra-tion measurements upon sinusoidal excitation. In Proceedings of the 13th FIG Symposium on Deformation Measurement and Analysis, Lisbon, Portugal, 12–15 May 2008. 69. Palazzo, D.R.; Friedmann, R.M.P.; Nadal, C.A.; Filho, M.L.S.; Veiga, L.A.K.; Faggion, P.L. Dynamic monitoring of structures using a robotic total station. In Proceedings of the 23th International FIG Congress, Munich, Germany, 8–13 October 2006. 70. Gikas, V.; Daskalakis, S. Full scale validation of tracking total stations using a long stroke electrodynamic shaker. In Proceedings of the 23th International FIG Congress, Munich, Germany, 8–13 October 2006. 71. Stempfhuber, W. Verification of the Trimble Universal Total Station (UTS) Performance for Kinematic Applications. In Proceedings of the Conference on Optical 3-D Measurement Techniques, Vienna, Austria, 1–3 July 2009. 72. Marendi´c,A.; Paar, R.; Damjanovi´c,D. Measurement of bridge dynamic displacements and natural frequencies by RTS. Gradev. J. Croat. Assoc. Civ. Eng. 2017, 69, 281–294. [CrossRef] 73. Roberts, G.W.; Meng, X.; Dodson, A.H. Integrating a Global Positioning System and Accelerometers to Monitor the Deflection of Bridges. J. Surv. Eng. 2004, 130, 65–72. [CrossRef] 74. Yu, J.; Meng, X.; Shao, X.; Yan, B.; Yang, L. Identification of dynamic displacements and modal frequencies of a medium-span suspension bridge using multimode GNSS processing. Eng. Struct. 2014, 81, 432–443. [CrossRef] Appl. Sci. 2021, 11, 3893 29 of 29

75. Meng, X.; Nguyen, D.; Xie, Y.; Owen, J.; Psimoulis, P.; Ince, S.; Chen, Q.; Ye, J.; Bhatia, P. Design and Implementation of a New System for Large Bridge Monitoring—GeoSHM. Sensors 2018, 18, 775. [CrossRef] 76. Kaloop, M.; Elbeltagi, E.; Hu, J.; Elrefai, A. Recent Advances of Structures Monitoring and Evaluation Using GPS-Time Series Monitoring Systems: A Review. ISPRS Int. J. Geo-Inf. 2017, 6, 382. [CrossRef] 77. Yu, J.; Meng, X.; Yan, B.; Xu, B.; Fan, Q.; Xie, Y. Global Navigation Satellite System-based positioning technology for structural health monitoring: A review. Struct. Control Heal. Monit. 2020, 27.[CrossRef] 78. Paar, R. Geospatial Data Bases of Objects in the Highway Management System of the Republic of Croatia. Ph.D. Thesis, University of Zagreb, Zagreb, Croatia, 2010. 79. Ehrhart, M.; Lienhart, W.; Kalenjuk, S. Monitoring of Bridge vibrations with image-assisted total stations. In Proceedings of the 4th Conference on Smart Monitoring, Assessment and Rehabilitation of civil Structures (SMAR), Zurich, Switzerland, 13–15 September 2017. 80. Ehrhart, M.; Lienhart, W. Image-based dynamic deformation monitoring of civil engineering structures from long ranges. In Proceedings of the SPIE 9405, Image Processing: Machine Vision Applications VIII, San Francisco, CA, USA, 10–11 February 2015. 81. Paar, G. Deformation Monitoring by High Resolution Terrestrial Long Range Sensing. 2014. Available online: https://cordis. europa.eu/project/id/285839 (accessed on 25 March 2021). 82. Wagner, A.; Huber, B.; Wiedemann, W.; Paar, G. Long-Range Geo-Monitoring Using Image Assisted Total Stations. J. Appl. Geod. 2014, 8.[CrossRef] 83. Paar, R.; Marendi´c,A.; Wagner, A.; Wiedemann, W.; Wunderlich, T.; Roi´c,M.; Damjanovi´c,D. Using IATS and digital levelling staffs for the determination of dynamic displacements and natural oscillation frequencies of civil engineering structures. In Proceedings of the INGEO 2017—7th International Conference on Engineering Surveying, Lisbon, Portugal, 18–20 October 2017.