applied sciences
Article Real-Time Tunnel Deformation Monitoring Technology Based on Laser and Machine Vision
Zurong Qiu * , Haopeng Li , Wenchuan Hu, Chenglin Wang, Jiachen Liu and Qianhui Sun
State Key Laboratory of Precision Measuring Technology and Instruments, School of Precision Instruments and Opto-electronics Engineering, Tianjin University, Tianjin 300072, China; [email protected] (H.L.); [email protected] (W.H.); [email protected] (C.W.); [email protected] (J.L.); [email protected] (Q.S.) * Correspondence: [email protected]
Received: 23 October 2018; Accepted: 7 December 2018; Published: 11 December 2018
Featured Application: The proposed tunnel deformation monitoring system was designed for tunnel during construction in dusty and dark environment with low visibility, and it also can be applied for the tunnel that has already been built.
Abstract: Structural health monitoring is a topic of great concern in the world, and tunnel deformation monitoring is one of the important tasks. With the rapid developments in tunnel traffic infrastructure construction, engineers need a portable and real-time system to obtain the tunnel deformation during construction. This paper reports a novel method based on laser and machine vision to automatically measure tunnel deformation of multiple interest points in real time and effectively compensate for the environment vibration, and moreover it can overcome the influence of a dusty and dark tunnel environment in low visibility. An automatic and wireless real-time tunnel deformation monitoring system, which is based on laser and machine vision and can give early warnings for tunnel collapse accidents, is proposed. The proposed system uses a fixed laser beam as a monitoring reference. The image acquisition modules mounted on the measured points receive the laser spots and measure the tunnel accumulative deformation and instantaneous deformation velocity. Compensation methods are proposed to reduce measurement errors caused by laser beam feasibility, temperature, air refraction index, and wireless antenna attitude. The feasibility of the system is verified through tunnel tests. The accuracy of the detection system is better than 0.12 mm, the repeatability is less than 0.11 mm, and the minimum resolution is 10 µm; therefore, the proposed system is very suitable for real-time and automatic detection of tunnel deformation in low visibility during construction.
Keywords: structural health monitoring; real-time monitoring; tunnel deformation measurement; machine vision; laser beam; wireless; low visibility
1. Introduction Structural health monitoring plays an indispensable role in diagnosing structure safety around the world [1], and tunnel deformation monitoring is one of the important tasks [2]. With the continuous expansion of the scale of tunnel traffic infrastructure construction, tunnel collapse accidents have occurred frequently in recent years [3]; this indicates that the traditional methods for monitoring tunnel deformation are unable to meet the new safety monitoring requirements for large tunnel traffic infrastructure [4], when real time becomes an important issue. In tunnel construction, the tunnel excavation surface is composed of tunnel face, side face, and roof face. Numerous workers are present on the excavation surface with dusty and dark environment in low visibility [5], and the tunnel
Appl. Sci. 2018, 8, 2579; doi:10.3390/app8122579 www.mdpi.com/journal/applsci Appl. Sci. 2018, 8, 2579 2 of 23 deformation is at a maximum; thus, monitoring the deformation of this fracture surface in real time is important. The current tunnel deformation monitoring technology, including non-machine vision measurement methods and machine vision measurement methods [6], however, is not effective enough. Most of the existing equipment is manually operated and susceptible to a dusty and dark environment in low visibility, and cannot meet real-time automatic monitoring and early warning requirements or provide timely warnings about dangerous deformation [7]. Non-machine vision measurement methods are widely used in tunnel deformation monitoring; however, they are susceptible to environmental and human factors, and take a long time. Tape extensometers and convergence gauge are unstable and operated manually; in addition, they are strongly influenced by environmental and human factors [8]. The leveling method mainly addresses the tunnel settlement value [9] and requires many monitoring points to be set; thus, the error accumulates with the increase in the tunnel length. The indoor global positioning system measurement method requires many reference signal transmitters; however, it is difficult to ensure the stability of the reference signal transmitters position. The traditional total station measurement method has a low degree of automation, exhibits strong subjectivity, and requires a fixed station; these features affect the tunnel construction period [10]. The new-generation total station measurement method allows stations to be freely monitored, compared with the traditional method [11,12], but it cannot be used to monitor all kinds of tunnels because of its high cost [13]. The fiber Bragg grating measurement method [14] can be used to monitor tunnel deformation constantly over a long period of time, but it is expensive, and the relevant equipment is easily damaged. Moreover, the equipment needs to be embedded into the structure or adhered onto the structure surface during monitoring, which results in a higher demand on the environment. A deformation monitoring system based on ultrasonic sensors [15] is automatic and in real time; however, it is only suitable for small- to middle-size tunnels, and it cannot be used to realize early warnings of dangerous deformation during the construction. Terrestrial laser scanning [16] achieves tunnel deformation monitoring; however, it takes a long time to obtain and process point cloud data, and cannot realize real-time monitoring. A machine vision measurement method for tunnel deformation develops a lot in the last few years [17]; however, existing machine vision measurement methods are not suitable for real-time tunnel deformation monitoring during construction with dusty and dark environments. A three-dimensional reconstruction method based on machine vision [18] needs to take lots of photos and the data processing algorithms are complex; this takes a long time and cannot meet real-time automatic monitoring and early warning requirements. A digital close-range photogrammetric method [19] is susceptible to tunnel dust and light intensity, heavy dust and a dark environment during construction reduces photo quality and detection accuracy. A photogrammetric method for tunnel detection based on a CCD camera [20] has a higher requirement for brightness and lighting equipment; this requires external power supply; when using batteries, the equipment is heavy and not easy to install; when using an electric generator, extra great vibration is produced and this affects the result. The infrared photogrammetric method [21] reduces dust impact at the price of decreasing photo brightness; as a result, the detection accuracy is low and cannot be used for real-time monitoring. Photogrammetric techniques for monitoring tunnel deformation [22] are only suitable for tunnels without dust after excavation, and it is also susceptible to dusty and dark environments. A tunnel detection method combining photogrammetry and laser scanning [23] is susceptible to the set position of the equipment; when the equipment is set deviating from the tunnel axis, the detection error increases obviously [24]; this indicates that the method is not suitable for a tunnel during construction, since the set position has a conflict with tunnel construction. This paper presents a real-time and automatic tunnel deformation monitoring system to realize real-time monitoring and early warning for dangerous tunnel deformation. Considering the dusty and dark tunneling environment, a fixed laser beam is used to transmit the monitoring reference from a stable point to a measured point through the dust and darkness in low visibility, and a shielded image acquisition module mounted on the measured points is used to acquire the laser spot image regardless Appl.Appl. Sci. Sci.2018 2018, 8, ,8 2579, x FOR PEER REVIEW 33 of of 23 23
the measurement result is affected by laser beam feasibility, temperature, air refraction index, and of the dust and darkness. However, because of the adverse tunnel environment, the measurement wireless antenna attitude. These factors are carefully examined in this study, and compensation result is affected by laser beam feasibility, temperature, air refraction index, and wireless antenna methods are proposed to reduce the measurement errors caused by them. The results show that the attitude. These factors are carefully examined in this study, and compensation methods are proposed proposed system not only achieves good detection accuracy, repeatability and resolution, but also to reduce the measurement errors caused by them. The results show that the proposed system not compensates for the shortcomings of the above methods, such as the low degree of automation, the only achieves good detection accuracy, repeatability and resolution, but also compensates for the lack of real-time measurement function and overcoming the influence of dusty and dark shortcomings of the above methods, such as the low degree of automation, the lack of real-time environments in low visibility. measurement function and overcoming the influence of dusty and dark environments in low visibility.
2.2. Methodology Methodology TheThe tunnel tunnel collapse collapse process process is is the the deformation deformation of of the the construction constructionfracture fracture surface surfacein inthe the tunnel,tunnel, whichwhich takestakes a fixed fixed point point of of the the earth earth as asa referenc a reference.e. At present, At present, the new the Austrian new Austrian tunneling tunneling method method[25], which [25], includes which includes designing designing an initial an support initial support and secondary and secondary lining to lining the construction to the construction fracture fracturesurface surface[26], is [mostly26], is mostlyadopted adopted in tunnel in tunnelconstruc construction.tion. The secondary The secondary lining liningis constructed is constructed on the onbasis the basisof the of initial the initial support support when when the thedeformation deformation of ofthe the initial initial support support becomes becomes stable [[27].27]. Therefore,Therefore, the the deformation deformation of of the the secondary secondary lining lining surface surface can can be be ignored; ignored; however, however, the the deformation deformation ofof the the excavation excavation surface surface and and the the edge edge part part of of the the initial initial support support is is significant. significant. The The proposed proposed system system takestakes a a fixed fixed point point of of the the earth earth as as monitoring monitoring reference reference and and measures measures the the deformation deformation of of excavation excavation surface.surface. Given Given the the good good directivity directivity and and energy energy concentration concentration of of a a laser, laser, as as well well as as the the dusty dusty and and dark dark tunnelingtunneling environmentenvironment inin lowlow visibility,visibility, a a laser laser beam beam is is used used to to transmit transmit the the monitoring monitoring reference reference fromfrom the the fixed fixed point point to theto the measured measured point point through thro theugh dust the anddust darkness, and darkness, thereby thereby forming forming an optical an projectionoptical projection of the fixed of the point fixed on point the intersecting on the intersec surfaceting nearsurface the near measured the measured point, which point, is which the optical is the equivalentoptical equivalent reference, reference, to monitor to themonitor tunnel the deformation tunnel deformation in the system. in the A system. sketch mapA sketch of the map system of the is shownsystem in is Figure shown1. in Figure 1.
Figure 1. System principle. Figure 1. System principle. A laser transmitter module, whose laser beam points to the intersecting surface near the measured pointsA and laser forms transmitter an optical module, projection whose of laser the fixed beam point points as to a laserthe intersecting spot, is firmly surface fixed near on thethe earth.measured To obtain points the and position forms relationshipan optical projection between of the the measured fixed point point as anda laser the spot, fixed is point, firmly an fixed image on acquisitionthe earth. To module obtain mounted the position on the relationship measured point between and locatedthe measured perpendicularly point and to the the fixed laser point, beam isan usedimage to acquisition acquire the lasermodule spot mounted image of on the the laser measured beam. This point module and located monitors perpendicularly the laser spot positionto the laser in thebeam image is used coordinate to acquire system the withlaser machine spot image vision of th technology.e laser beam. The This image module acquisition monitors mode the is shownlaser spot in Figureposition2. Thein the image image sensor coordinate acquires system the laser with spot machine image through vision technology. a laser receiving The image screen acquisition placed in frontmode of is the shown sensor. in Figure 2. The image sensor acquires the laser spot image through a laser receiving screen placed in front of the sensor. Appl. Sci. 2018, 8, x FOR PEER REVIEW 4 of 23 Appl. Sci. 2018, 8, 2579 4 of 23 Appl. Sci. 2018, 8, x FOR PEER REVIEW 4 of 23
Figure 2. Image acquisition module. Figure 2. Image acquisition module.module. The image acquisition module is fixed on the measured points. When the excavation surface sinksThe during imageimage acquisitionacquisitionthe deformation, modulemodule the isis fixedfixedimage onon acquisition thethe measuredmeasured module points.points. will WhenWhen sink thethe along excavationexcavation with it, surfacesurface thereby sinkschanging duringduring the the deformation,laser deformation, spot position the the image inimage acquisitionthe imageacquisition modulecoordinate module will system. sink will along sinkThe with alongchange it, therebywith of theit, changing laserthereby spot thechangingposition laser spot the in the positionlaser image spot in coordinate theposition image insystem coordinate the imagereflects system. c oordinatethe deformation The changesystem. of of theThe the excavation change laser spot of surface, positionthe laser including in spot the imagepositionaccumulative coordinate in the image deformation system coordinate reflects and system theinstantaneous deformation reflects thdeformation ofe deformation the excavation velocity of the surface, [28]. excavation including surface, accumulative including deformationaccumulativeThe mathematical and deformation instantaneous model and instantaneous deformation of this principle velocity deformation is shown [28]. in velocity Figure [28].3. The geometric center of the laser is Thethe coordinate mathematical origin, modelmodel the ofof tunnel thisthis principleprinciple driving isis direction shown in is Figure the z3 -axis,3.. TheThe geometricthegeometric horizontal centercenter direction ofof thethe laserlaser is the isis thexthe-axis, coordinatecoordinate and the origin, verticalorigin, the theaxis tunnel tunnel is the driving drivingy-axis. direction Therefore,direction is the isa zthree-dimensionalthe-axis, z-axis, the horizontal the horizontal world direction coordinatedirection is the xis -axis,system the andx-axis,o-xyz the and is vertical established. the vertical axis is The theaxis ylaser -axis.is the beam Therefore,y-axis. points Therefore, ato three-dimensional the laser a three-dimensional receiving world screen coordinate andworld forms coordinate system a laser ospotsystem-xyz Pis on established.o-xyzthe isscreen. established. The laser The beam laser points beam to points the laser to receivingthe laser receiving screen and screen forms and a laser forms spot a Plaseron thespot screen. P on the screen.
FigureFigure 3. Mathematical3. Mathematical model. model.
Considering the horizontal and verticalFigure 3.position Mathematical information, model. the coordinates of the spot P in the Considering the horizontal and vertical position information, the coordinates of the spot P in world coordinate system o-xyz are (x, y). To examine the position information of laser spot P in the theConsidering world coordinate the horizontal system o-xyz and arevertical , position . To examine information, the position the coordinates information of of the laser spot spot P inP in image coordinate system, an image coordinate system on the laser receiving screen is established, as thethe world image coordinate coordinate system system, o-xyz arean image , . Tocoordina examinete thesystem position on informationthe laser receiving of laser spot screen P in is shown in Figure3. The o0x0 axis is parallel to the ox axis, and the o0y0 axis is parallel to the oy axis; the theestablished, image coordinate as shown system, in Figure an 3.image The ′ ′coordina axis teis systemparallel0 onto thethe laser axis, receiving and the screen ′ ′ axis is is coordinates of laser spot P in the image coordinate system are (x , y0). The coordinate transformation established,parallel to asthe shown axis; in theFigure coordinates 3. The ′ ′of laser axis spotis parallel P in the to image the coordinate axis, and system the ′ ′ are axis , ′is . of spot P in the two coordinate systems is shown in Equation (1). The constants c and c are coordinate parallelThe coordinate to the transformationaxis; the coordinates of spot ofP inlaser the spottwo coordinateP in the image systems coordinate is shown0 system in1 Equation are (1). , ′ The. conversion parameters, which are related to the relative position of the two coordinate systems: Theconstants coordinate transformationand are coordinate of spot conversionP in the two parameters, coordinate whichsystems are is relatedshown toin theEquation relative (1). position The constantsof the two coordinateand are systems: coordinate conversion( 0 parameters, which are related to the relative position x = x + c0 of the two coordinate systems: 0 = + . (1) y = y + c1 . (1) = + = + . (1) InIn the the three-dimensional three-dimensional world world coordinate coordinate = + system system o -o-xyzxyz,, thethe laserlaser beambeam equationequation is shown shown as as Equation (2), and the equation of the intersecting surface near the measured point is shown as EquationIn the three-dimensional(2), and the equation world of coordinatethe intersecti systemng surface o-xyz, thenear laser the beammeasured equation point is isshown shown as as Equation (3), where A, B, C, and d are constant parameters and are related to the laser position: EquationEquation (2), (3), and where the Aequation, B, C, and of d the are intersecticonstant parametersng surface nearand arethe related measured to the point laser isposition: shown as Equation (3), where A, B, C, and d are constant parameters and are related to the laser position: A × x× = × = × = B × y = C × z,, (2)(2) × = × = × = . , (2)(3) z = d. (3) According to Equations (1)–(3), the coordinates = . of the laser spot in the image coordinate system(3) o′-xAccording′y′ can be obtainedto Equations as shown (1)–(3), in the Equation coordinates (4): of the laser spot in the image coordinate system o′-x′y′ can be obtained as shown in Equation (4): Appl. Sci. 2018, 8, 2579 5 of 23
According to Equations (1)–(3), the coordinates of the laser spot in the image coordinate system o0-x0y0 can be obtained as shown in Equation (4):
( 0 C×d x = A + c0 C×d . (4) y0 = B + c1
According to Equation (4), when the positions of laser and image acquisition module are invariant, that is, when the parameters A, B, C, and d are invariant, the coordinates of the laser spot in the image C×d C×d coordinate system A + c0, B + c1 remain fixed. Therefore, the monitoring reference can be transmitted from the fixed point of the earth to the measured points. When the image acquisition module sinks during tunnel deformation, the image coordinate system moves relative to the world coordinate system o-xyz. Supposing that the vertical and horizontal deformation amounts of the image acquisition module are ∆x and ∆y, respectively, the laser spot position in the image coordinate system also shifts ∆x and ∆y in the vertical and horizontal directions, respectively. Therefore, when the image acquisition module shifts in the vertical and horizontal directions along with the measured point, the change of the laser spot position in the image coordinate system reflects the deformation. In an actual situation, the laser beam is not absolutely perpendicular to the laser receiving screen. As a result, when the image acquisition module moves along the z-axis with the measured point, the laser spot will produce the displacement component in the vertical and horizontal directions. As shown in Figure3, α is the angle between the laser beam and the ox axis, β is the angle between the laser beam and the oy axis, and θ is the angle between the laser beam and the oz axis. Assuming that the displacement change of image acquisition module along the oz axis is ∆z, the errors of the horizontal and vertical deformation measurements are as shown in Equation (5):
( cosα ∆x = ∆z × cosθ cosβ . (5) ∆x = ∆z × cosθ
Assuming that the laser beam points obliquely to the receiving screen, the angles α = 73.3◦, β = 81.5◦, and θ = 18.9◦ (in fact, the tilt of the laser beam is not that particularly obvious). When the image acquisition module shifts 1 mm along the oz axis, ∆x = 0.3037 mm and ∆y = 0.1562 mm. Therefore, when the shifting displacement of the image acquisition module along the z-axis is not large (and in fact it is not large) [29], this error can be ignored. The laser transmitter module is mounted on the fixed point of the earth distant from the construction site, and it should remain stable. However, in cases of emergency during construction, such as sudden vibrating, when the laser beam reference is shifted and rotated, the position of the laser spot in the image coordinate will change. Taking the x-axis coordinate as an example, if the displacements along the x-axis and the horizontal rotation angle of the laser beam reference are ∆x and ∆α, respectively, the resulting laser spot position error ∆x0 is as shown in Equation (6):
∆x0 = ∆x + L × tan∆α = ∆x + L × ∆α. (6)
In order to reduce this error, a data compensation method is proposed here. The data compensation method uses two backward lasers in the laser transmitter module and two fixed distant reference monitoring image acquisition modules with higher stability to monitor the feasibility of the laser transmitter, as shown in Figure4, where Ls represents the distance between the laser transmitter module and the image acquisition modules, ∆α represents the horizontal rotation angle of the laser transmitter module, and ∆x represents the displacement along the x-axis of the laser transmitter module. Appl. Sci. 2018, 8, 2579 6 of 23 Appl. Sci. 2018, 8, x FOR PEER REVIEW 6 of 23
Figure 4. Sketch map. 1. reference monitoring image acquisition module A; 2. reference monitoring Figure 4. Sketch map. 1. reference monitoring image acquisition module A; 2. reference monitoring image acquisition module B; 3. laser transmitter module; 4. image acquisition module. image acquisition module B; 3. laser transmitter module; 4. image acquisition module. Given that Ls were dozens of meters, small displacement and rotation angle will have no effect Given that Ls were dozens of meters, small displacement and rotation angle will have no effect on these Ls; that is, the lengths of Ls were invariable before and after the position change of the laser on these Ls; that is, the lengths of Ls were invariable before and after the position change of the laser transmitter module. With the two reference monitoring image acquisition modules, two laser spot transmitter module.0 With the0 two reference monitoring image acquisition modules, two laser spot position errors ∆x1 and ∆x2 can be obtained. The image acquisition module detecting displacement position errors 0 ∆ ′ and ∆ ′ can be obtained. The image acquisition module detecting with error is ∆x3. The equations of two variables ∆x and ∆α were obtained, as shown in Equation (7): displacement with error is ∆ ′ . The equations of two variables ∆ and ∆ were obtained, as shown in Equation (7): ( 0 ∆x1 = ∆x + L1 × ∆α 0 . (7) ∆ ′∆x =∆ + = ∆x + L ×∆ × ∆α 2 2 . (7) ∆ ′ =∆ + ×∆ According to Equation (10), ∆x and ∆α can be determined as shown in Equation (8): According to Equation (10), ∆x and ∆α can be determined as shown in Equation (8):
∆ ∆ x 0∆− ∆ x0 ∆∆ x = ∆∆ x 0 −− L ×× 1 2 1 1 L 1 −L2 0 0 . (8) ∆x1−∆x2 . (8) ∆ α =∆ ∆ = L 1 − L 2 Then,Then, the the detected detected displacement displacement can can be be compensated compensated in in Equation Equation (9): (9):