Deep Learning Correction Algorithm for the Active Optics System

sensors

Article Deep Learning Correction Algorithm for The Active Optics System

Wenxiang Li 1 , Chao Kang 1, Hengrui Guan 1, Shen Huang 2, Jinbiao Zhao 2, Xiaojun Zhou 1,2,* and Jinpeng Li 2

1 Nanjing Astronomical Instruments Research Center, University of Science and Technology of China, Hefei 230026, China; [email protected] (W.L.); [email protected] (C.K.); [email protected] (H.G.) 2 CAS Nanjing Astronomical Instruments Co., Ltd., Nanjing 210042, China; [email protected] (S.H.); [email protected] (J.Z.); [email protected] (J.L.) * Correspondence: [email protected]

Received: 14 August 2020; Accepted: 6 November 2020; Published: 9 November 2020

Abstract: The correction of wavefront aberration plays a vital role in active optics. The traditional correction algorithms based on the deformation of the mirror cannot eﬀectively deal with disturbances in the real system. In this study, a new algorithm called deep learning correction algorithm (DLCA) is proposed to compensate for wavefront aberrations and improve the correction capability. The DLCA consists of an actor network and a strategy unit. The actor network is utilized to establish the mapping of active optics systems with disturbances and provide a search basis for the strategy unit, which can increase the search speed; The strategy unit is used to optimize the correction force, which can improve the accuracy of the DLCA. Notably, a heuristic search algorithm is applied to reduce the search time in the strategy unit. The simulation results show that the DLCA can eﬀectively improve correction capability and has good adaptability. Compared with the least square algorithm (LSA), the algorithm we proposed has better performance, indicating that the DLCA is more accurate and can be used in active optics. Moreover, the proposed approach can provide a new idea for further research of active optics.

Keywords: active optics; wavefront aberrations; deep learning; correction algorithms; heuristic search

1. Introduction Active optics, a key technique in large modern telescopes, is applied to correct wavefront aberrations and reduce the influence of the primary mirror’s deformations on the beam quality [1]. Nowadays, active optics technology is widely utilized in the large modern telescope: The 8 m class VLT (Very Large Telescope) [2], the VISTA (Visible and Infrared Telescope for Astronomy) [3], and LSST (Large Synoptic Survey Telescope) [4]. The wavefront aberration of the telescope comes from multiple sources: atmospheric turbulence, manufacturing and assembly errors, and gravity deformation. The atmospheric wavefront aberrations are usually corrected by adaptive optics (AO), while the gravitational wavefront aberrations are corrected by active optics. Active optics detects the wavefront aberrations by the wavefront detection device such as the interferometer, the Shack–Hartmann sensor, and so on. Then the wavefront aberrations are compensated by using actuators at the back of the mirror, which can significantly improve the observation performance of the large telescopes. The active optics can be used not only in telescopes but also in the large-aperture standard mirror for optical detection. The large-aperture standard mirrors using active optics can work in different conditions. Therefore, the method to

Sensors 2020, 20, 6403; doi:10.3390/s20216403 www.mdpi.com/journal/sensors Sensors 2020, 20, 6403 2 of 15 improve the correction capability of the mirror has a great significance for the development of modern astronomy and optics. The calculation of the correction force is one of the key points to improve the correction capability of the mirror. Currently, the existing algorithms for active correction can be classified into three parts, including the algorithm based on free-vibration mode [5], bending mode [6], and Zernike mode [7]. The first kind of algorithm is based on free-vibration mode to solve the correction force of each support and the free-vibration surface shape of the mirror, then the correction target can be fitted with the free-vibration surface shape. Finally, the correction force can be obtained. However, it only reflects the intrinsic characteristics of the mirror, which may not be fully realized when the lateral support is included [8]. The second kind of algorithm is based on bending mode. The stiffness matrix of the mirror can be transformed into a series of orthogonal surfaces shape with the same RMS (Root Mean Square), but this approach relies on manual analysis and selection, which reduces the efficiency of correction [9]. The third kind of algorithm is based on Zernike mode to calculate the correction force, in which the least square algorithm is the most widely used [10]. The least square algorithm is efficiently to correct wavefront aberrations based on the response mirror shape of each active support and obtain good performance in actual application. For example, Dai et al. [11] used the least square algorithm to correct the gravitational deformations of a 1.2 m thin mirror and studied the capability of fitting Zernike aberrations. Zhou et al. [12] applied the least square algorithm to fit the correction target, which improving the observation performance of space telescopes. Schwaer et al. [13] utilized the least square algorithm to calculate the correction force to analyze the support system of a lightweight, low-cost telescope. However, the above three types of algorithms strictly depend on the deformation of the mirror, and cannot effectively deal with the various disturbances, such as thermal distortion, and gravity deformation, especially the error in structure and actuator control. These disturbances will bring great influences on the correction capability of the mirror. The system identification of the neural network can be utilized in the active optics because the neural network can establish the mapping of the systems with disturbances [14,15]. However, the shallow neural network is prone to be affected by the problem of local optimum during training, so the shallow neural network cannot effectively describe systems with disturbances. The development of deep learning in recent years has brought inspiration for solving this problem. The model of the mirror can be replaced by the deep neural network, and the system identification can be transformed into the parameter optimization of the deep neural network. Some studies have noticed the application of deep learning in the field of optics. Guo et al. [16] have used an improved deep convolutional neural network to replace the traditional gradient-based optimization methods, whose idea is to establish the nonlinear mapping between the input point spread functions and the corresponding phase maps of the optical system. Hegde [17] has employed deep learning for optics design optimizations. Using deep learning indirectly to choose initializations and candidate preselection. The big datasets and long-time epochs are no longer needed. Gomez et al. [18] have applied the convolutional neural network to propose a reconstruction algorithm that can correct the image aberrations. Xu et al. [19] have established a control model based on deep learning for adaptive optics systems, the system control model is fitted with a deep neural network. Compared with the traditional control mode, the deep learning control model achieves better performance. All of these applications have brought new enlightenment for active optics. In this study, to maintain the good shape of a large-aperture standard mirror under different working conditions, active optics is applied to correct the deformation of the mirror [20]. A deep neural network is developed for a new correction algorithm in the active optics system. This deep learning correction algorithm combines with an actor network and a strategy unit. The actor network aims to fit the mapping of the active optics system with disturbances, minimize the wavefront aberrations, and optimize the correction force. However, it is hard to find the optimal force. The strategy unit is added to optimize the force. The correction force output by the actor network is used as the search Sensors 2020, 20, 6403 3 of 15

Sensors 2020, 20, 6403 3 of 16 basis of the strategy unit, and then corresponding strategies are input into the active optics system to utilizedpick out in the the optimal strategy correction unit to further force. Moreover, improve th ae heuristic search speed. search The algorithm proposed is utilized integrated in the algorithm strategy canunit effectively to further improve thethe searchcorrection speed. capability The proposed to deal integratedwith disturbances. algorithm can eﬀectively improve the correctionThis paper capability is organized to deal as withfollows. disturbances. Section 2 illustrates the used materials and the principle of the deepThis learning paper is organizedcorrection asalgorithm follows. (DLCA) Section 2in illustrates this study. the In used Section materials 3, we andstudy the the principle correction of performancethe deep learning of the correction DLCA and algorithm compare (DLCA) the traditio in thisnal study.correction In Section algorithm3, we by study conducting the correction a series ofperformance experiments. of the Some DLCA discussions and compare about the traditionalthe proposed correction algorithm algorithm are given by conducting in Section a series4. The of conclusionsexperiments. are Some drawn discussions in Section about 5. the proposed algorithm are given in Section4. The conclusions are drawn in Section5. 2. Materials and Methods 2. Materials and Methods 2.1. Experimental Materials and Data Collection 2.1. Experimental Materials and Data Collection

2.1.1. Simulation Simulation Environment Environment To validate the correction performance of the DLCA, an active optics platform is is established. established. This platform using the finite ﬁnite element analysis software (ANSYS) is based on a 1 m m diameter diameter standard standard mirror. The key components of the standardstandard mirror are shownshown inin TableTable1 1..

TableTable 1. The key components of the standard mirror.

NameName Parameter Parameter Diameter 1000 mm Diameter 1000 mm ThicknessThickness 80 80 mm mm RadiusRadius of of Curvature Curvature 4000 4000 mm mm MaterialMaterial K9 K9 Glass Glass MassMass 174.445 174.445 kg kg

The support system of the active optics platform is composed of 24 axial supports and 3 lateral supports. The 2424 axialaxial supportssupports that that include include 21 21 active active supports supports and and 3 fixed3 fixed supports supports on on the the back back of the of themirror, mirror, which which is distributed is distributed on on three three rings rings whose whose radiuses radiuses are are 121.88 121.88 mm mm (3 (3 supports),supports), 304.86304.86 mm (6 supports),supports), andand 444.8 444.8 mm mm (12 (12 support). support). The The fixed fixed supports supports are are distributed distributed on theon secondthe second support support ring ringto constrain to constrain the three the degrees three ofdegrees freedom ofR xfreedom, Ry, Rz of the , mirror. , The of gravitationalthe mirror. deformationsThe gravitational of the deformationsmirror tend to of be the more mirror sensitive tend to thebe more axial sensit supports,ive to so the the axial deformation supports, caused so the bydeformation lateral supports caused is bymuch lateral less thansupports the axial is much supports. less Therefore,than the theaxial lateral supports. supports Therefore, are all composed the lateral of passivesupports supports, are all whichcomposed are onlyof passive used tosupports, balance thewhich gravity are only under used diff toerent balance working the gravity conditions. under The different support working system conditions.and finite element The support analysis system (FEA) and are finite shown element in Figure an1alysis. The (FEA) active are supports shown are in markedFigure 1. from The 1 active to 21 supportsin a clockwise are marked direction. from The 1 to support 21 in a clockwise points are direction. all indicated The by support red arrows. points are all indicated by red arrows.

Figure 1. TheThe support support system system of of the standard mirror and finite ﬁnite element analysis.

Sensors 2020, 20, 6403 4 of 15

2.1.2. Generation of Datasets In the active optics system, the mirror strictly conforms to Hooke’s law of linearity, which mainly has two aspects:

The same force always produces the same mirror’s deformation, which is independent of the • initial shape of the mirror; that is, the displacement of any point of the mirror is linearly related to the force of the actuator. The deformations of the mirror conform to the linear superposition of forces. • Therefore, for the single actuator, the relationship between the deformation of the mirror and the force is linear. The mirror’s deformation can be calculated as

Wi(x, y) = Fiwi(x, y) (1) where Wi(x, y) is the deformation of the mirror after Fi caused by the i-th actuator, Fi is the force exerted by the i-th actuator, wi(x, y) is the mirror’s deformation caused by the unit force applied by the i-th actuator, which is called the response function of the actuator, and i is the number of actuators. The matrix composes of each actuator’s response function is called the stiﬀness matrix of the mirror. The force of each actuator will cause the mirror’s deformation, and the total deformation is the linear superposition of the deformation caused by each actuator. So, the total deformation of the mirror can be expressed as Xn W(x, y) = Wi(x, y) (2) i=1 where W(x, y) is the total deformation of the mirror, Wi(x, y) is the mirror’s deformation caused by the i-th actuator, n is the number of actuators. Based on the above analysis, the mirror’s deformation and the correction force have the following equation in the active optic system [21–23]:

C f = W (3) − where C is a stiﬀness matrix, f is the correction force, W is the mirror’s deformation. When the random force is added, we have the following equation:

C( f + ∆ f ) = (W + ∆W) (4) − where ∆ f is the random force, ∆W is the mirror’s deformation caused by ∆ f . Then, Equation (5) can be obtained by Equations (3) and (4): C∆ f = ∆W (5) − Based on Equation (5), we can know the relationship between the deformation of the mirror and force, so we can get a large number of data. However, these data do not contain disturbances. To solve this problem, a random disturbance is added to the stiﬀness matrix to simulate the disturbance of the real system by MATLAB. Therefore, all the training data contain disturbances. In addition, in the real active optics system, it is easier to obtain the training set, because the input and output data already include a variety of disturbances.

2.2. Method

2.2.1. The Traditional Correction Algorithm and the DLCA In the active optics system, the correction algorithms are aimed at eliminating the wavefront aberrations. However, the traditional correction algorithms directly calculate the correction force based on the deformations of the mirror, as shown in Figure2a. Sensors 2020, 20, 6403 5 of 15

Sensors 2020, 20, 6403 5 of 16

(a)

(b)

FigureFigure 2. The 2. The diff erencedifference of structure of structure between between the the traditional traditional correction correction algorithms algorithms and and the the deep deep learning correctionlearning algorithm correction (DLCA). algorithm (a )(DLCA). The structure (a) The ofstructure the traditional of the traditional correction correction algorithm; algorithm; (b) The ( structureb) The structure of the DLCA. of the DLCA. In this study, the calculation of the correction force is based on input data and output data. In In this study, the calculation of the correction force is based on input data and output data. addition, a strategy unit based on feedback ideas is introduced to optimize the correction force output In addition, a strategy unit based on feedback ideas is introduced to optimize the correction force by the actor network. Then, the possible forces output by the strategy unit will be applied to the outputmirror, by theand actorthe RMS network. of the mirror Then, wi thell be possible measured forces by the output interferometer, by the strategy as shown unit in willFigure be 2b. applied In to the mirror,this way, and the thecorrection RMS of force the is mirror not calculated will be measureddirectly, and by it thewill interferometer, be optimized by as the shown strategy in unit, Figure 2b. In thiswhich way, effectively the correction improves force the iscorrection not calculated accuracy directly, of the DLCA. and it In will Figure be optimized 2b, the actor by network the strategy and unit, whichthe estrategyffectively unit improves have different the correction roles and the accuracy specific of contributions the DLCA. are In Figureas follows:2b, the actor network and the strategy• The actor unit havenetwork diff iserent developed roles and for quickly the specific outpu contributionstting the correction are as force follows: to provide a search basis for the strategy unit, which significantly reduces the search time, and the convergence The actor network is developed for quickly outputting the correction force to provide a search • speed of the DLCA is quickened. basis for the strategy unit, which significantly reduces the search time, and the convergence speed • The strategy unit is used to optimize the correction force output by the actor network to achieve of thehigher DLCA correction is quickened. accuracy. The strategy unit is used to optimize the correction force output by the actor network to achieve • 2.2.2.higher The correction Actor Network accuracy. The actor network adopts a deep neural network (DNN) [24]. The DNN can find out complex 2.2.2. The Actor Network structures in large datasets by adjusting internal parameters. These parameters use the neuron informationThe actor networkof the previous adopts layer a deep to calculate neural the network neuron (DNN) information [24]. of The the DNN next layer. can find Essentially, out complex structuresthe DNN in can large fit any datasets input-to-output by adjusting mapping internal with parameters. its advantage Theseof automatic parameters feature useextraction the neuron informationfrom data, of with the no previous need for layerany model to calculate by manual the design neuron [25–27]. information The DNN of is the composed next layer. of an Essentially,input layer, multiple hidden layers, and an output layer, which are all fully connected (FC). Starting from the DNN can fit any input-to-output mapping with its advantage of automatic feature extraction from the input layer, each layer transforms simple representations into more abstract representations until data,the with output no needlayer. forThe any parameters modelby of manualthe actor designnetwork [25 are–27 randomly]. The DNN initialized is composed with small of numbers an input layer, multiplebefore hidden the training. layers, The and process an output of the layer, actor whichnetwork are is allshown fully in connected Figure 3, and (FC). it includes Starting two from main the input layer,phases: each layer transforms simple representations into more abstract representations until the output layer. The parameters of the actor network are randomly initialized with small numbers before the training.Forward The Propagation process of Phase the actor network is shown in Figure3, and it includes two main phases: The deformation of the mirror is fitted into 65 Zernike coefficients, which are input to the actor Forwardnetwork Propagation as a mini-batch Phase to improve the speed of training. The information is transmitted forward layerThe by deformation layer, starting of thefrom mirror the input is fitted layer, into passing 65 Zernike through coetheffi hiddencients, layer which to the are output input tolayer. the actor networkThe actual as a mini-batch output of the to actor improve network the speedis estima ofted training. correction The force, information it can be iscalculated transmitted as forward layer Sensors 2020, 20, 6403 6 of 15 by layer, starting from the input layer, passing through the hidden layer to the output layer. The actual output of the actor network is estimated correction force, it can be calculated as

Sensors 2020, 20, 6403 6 of 16 fe = σn(( σ (σ (Wθ )θm ) )θmn) (6) ··· 2 1 m1 2 ··· where fe is the estimated correction = (( force,⋯ (((σ1,σ2, σ)3 )σ⋯n))are) activation functions for each(6) layer, ··· (θm1,θm2, θm3 θmn) are the network parameters for each layer, W is the deformation of the mirror. where is the··· estimated correction force, (,, ⋯) are activation functions for each layer, Backward(,, Propagation ⋯) are Phase the network parameters for each layer, is the deformation of the mirror.

BackwardThe output Propagation of the actorPhase network is divided into two types: actual output and ideal output. fe is the actual output, and the ideal output was f . Therefore, the loss function can be obtained as The output of the actor network is divided into two types: actual output and ideal output. is the actual output, and the ideal output was . Therefore,n the loss function can be obtained as 1 X 2 L(θm) = ( f f ) (7) 1 n i − ei ( )= (i=1− ) (7) where θm is the network parameters for each layer, i is the node order of the output layer, n is the where is the network parameters for each layer, is the node order of the output layer, n is the number of nodes in the output layer, L(θm) is the mean square error of f and fe, and its function is to number of nodes in the output layer, ( ) is the mean square error of and , and its function adjust the parameters of the actor network to make the prediction more accurate. In the backward is to adjust the parameters of the actor network to make the prediction more accurate. In the propagation phase, the optimizer is needed to optimize the loss function. An optimization algorithm backward propagation phase, the optimizer is needed to optimize the loss function. An optimization based on stochastic gradient descent called Adam [28], which can automatically adjust the learning algorithm based on stochastic gradient descent called Adam [28], which can automatically adjust the rate according to the value of the gradient and improve training eﬃciency. learning rate according to the value of the gradient and improve training efficiency.

FigureFigure 3. 3. TheThe process process of ofthe the actor actor network. network.

2.2.3.2.2.3. TheThe Strategy Unit Unit TheThe outputoutput of the actor network network is is the the estimated estimated force, force, which which is isnot not optimal. optimal. Therefore, Therefore, it is it is necessarynecessary to improve improve the the accuracy accuracy of ofthe the correction correction force. force. When When the estimated the estimated force isforce received, is received, the therandom random correction correction force force is added is added in its in field its fieldto obta toin obtain the correction the correction force in force all directions in all directions as much as as much aspossible. possible. Then Then the the correction correction force force is input is input into into the theactive active optics optics system system respectively respectively and andthe value the value of of RMSRMS cancan bebe detected by by the the interferometer. interferometer. Finally, Finally, the the search search will will be bestopped stopped when when the requirements the requirements areare met. met. However,However, thisthis aimlessaimless search method o occupiesccupies significant significant computing computing resources resources and and takes takes a a lot oflot time. of time. Therefore, the heuristic search algorithm is introduced to reduce the search time. The heuristic Therefore, the heuristic search algorithm is introduced to reduce the search time. The heuristic search is a kind of method based on the estimated force output by the actor network, but it will further search is a kind of method based on the estimated force output by the actor network, but it will further explore and find the optimal correction force. The heuristic search uses the existing information explore and find the optimal correction force. The heuristic search uses the existing information related related to the problem as a basis to improve search efficiency and reduce the search times. The heuristic function can be defined as

() =g() +h() (8) Sensors 2020, 20, 6403 7 of 15 to the problem as a basis to improve search eﬃciency and reduce the search times. The heuristic function can be deﬁned as f (n) = g(n) + h(n) (8) Sensors 2020, 20, 6403 7 of 16 where f (n) is the comprehensive priority of node n, g(n) is the cost of node n from the starting point, whereand h( n)(is) the is costthe comprehensive of node n from thepriority endpoint. of node When , theg( algorithm) is the cost is to of jump node to the from next the node starting to be point,searched, and the h( direction) is the cost that of can node make f from(n) be the the endpoint. largest is When always the selected. algorithm is to jump to the next nodeThe to be heuristic searched, search the direction includes that many can mature make algorithms,() be the suchlargest as is the always stochastic selected. parallel gradient descentThe (SPGD) heuristic algorithm search includes [29], the simulatedmany mature annealing algori algorithmthms, such [30 as,31 the], the stochastic ant colony parallel algorithm gradient [32], descentthe hill-climbing (SPGD) algorithm algorithm [29], [ 33the], simulated the genetic annealing algorithm algorithm [34], the [30,31], greedy the ant algorithm colony algorithm [35], and [32], the theevolutionary hill-climbing strategy algorithm algorithm [33], [36 the–38 ].genetic The evolutionary algorithm [34], strategy the algorithmgreedy algorithm is used to [35], accelerate and the evolutionarysearch in this strategy study due algorithm to the excellent [36–38]. performanceThe evolutiona ofry parallel strategy computing. algorithm is The used main to accelerate optimization the processessearch in this include study initialization, due to the excellent selection, performance mutation, crossover, of parallel and computing. evolution, The as shown main optimization in Figure4. processes include initialization, selection, mutation, crossover, and evolution, as shown in Figure 4.

Figure 4. FlowFlow chart chart for for the correction force search with evolutionary strategy algorithm.

2.2.4. The Working Procedure of the DLCA The working procedure procedure of of the the DLCA DLCA is isillustrated illustrated in inFigure Figure 5,5 ,and and it itincludes includes four four parts: parts: (1) (1)Generation Generation of datasets: of datasets: To Tovalidate validate the the feasibilit feasibilityy of ofthe the DLCA, DLCA, the the training training data data is is required required to establish the mapping of the activeactive opticsoptics systemsystem withwith thethe disturbance.disturbance. Based on the analysis of Section 2.1.2,2.1.2, the training data can be obtained effi eﬃciently. (2) The pre-training of the actor network: The actoractor networknetwork is is trained trained in in advance advance with with the the training training data data containing containing the th disturbance.e disturbance. Then, Then, the lossthe lossfunction function of the of network the network is established is established according according to the actualto the outputactual andoutput the idealand the output. ideal Theoutput. gradient The gradientdescent algorithm descent algorithm is used to minimizeis used to the minimize loss function the andloss updatefunction parameters. and update (3) Correctionparameters. force (3) Correctionoptimization: force The optimization: strategy unit The is utilized strategy to unit optimize is utilized the correction to optimize force, the and correction all possible force, strategies and all possibleare input strategies into the activeare input optics into system the active for validation.optics system When for validation. the optimal When correction the optimal force is correction obtained, forcethe input is obtained, of the actor the networkinput of the will actor be updated. network (4) will Application be updated. of (4) the Application DLCA: The of well-trained the DLCA: actor The well-trainednetwork is used actor to network determine is used the correction to determine force. the correction force.

Sensors 2020, 20, 6403 7 of 16 where () is the comprehensive priority of node , g() is the cost of node from the starting point, and h() is the cost of node from the endpoint. When the algorithm is to jump to the next node to be searched, the direction that can make () be the largest is always selected. The heuristic search includes many mature algorithms, such as the stochastic parallel gradient descent (SPGD) algorithm [29], the simulated annealing algorithm [30,31], the ant colony algorithm [32], the hill-climbing algorithm [33], the genetic algorithm [34], the greedy algorithm [35], and the evolutionary strategy algorithm [36–38]. The evolutionary strategy algorithm is used to accelerate the search in this study due to the excellent performance of parallel computing. The main optimization processes include initialization, selection, mutation, crossover, and evolution, as shown in Figure 4.

Figure 4. Flow chart for the correction force search with evolutionary strategy algorithm.

2.2.4. The Working Procedure of the DLCA The working procedure of the DLCA is illustrated in Figure 5, and it includes four parts: (1) Generation of datasets: To validate the feasibility of the DLCA, the training data is required to establish the mapping of the active optics system with the disturbance. Based on the analysis of Section 2.1.2, the training data can be obtained efficiently. (2) The pre-training of the actor network: The actor network is trained in advance with the training data containing the disturbance. Then, the loss function of the network is established according to the actual output and the ideal output. The gradient descent algorithm is used to minimize the loss function and update parameters. (3) Correction force optimization: The strategy unit is utilized to optimize the correction force, and all

Sensorspossible2020 strategies, 20, 6403 are input into the active optics system for validation. When the optimal correction8 of 15 force is obtained, the input of the actor network will be updated. (4) Application of the DLCA: The well-trained actor network is used to determine the correction force.

Figure 5. The working procedure of the DLCA.

3. Results

3.1. Determination of Network Parameters and Datasets Update

3.1.1. The Parameters Setting of the Actor Network To construct and test the actor network, the 5000 collected data were segregated into a training set, a validation set, and a testing set in a manner consistent with a 3:1:1 ratio [39]. All three sets of data have diﬀerent roles. The training set is used to identify parameters and constructs the actor network, the validation set is applied to evaluate the actor network, and the testing set is utilized to judge the ﬁnal performance of the actor network. Detailed parameter settings of the actor network are listed in Table2.

Table 2. The parameter setting of the actor network.

FC1 FC2 FC3 FC4 FC5 Activation Learning Optimizer Nodes Nodes Nodes Nodes Nodes Functions Rate 65 145 200 120 21 ReLU Adam 0.001

3.1.2. The Configuration of the Strategy Unit To improve the correction accuracy of the active optics system, the evolutionary strategy algorithm mentioned above was applied to reduce the search time. The 21 correction forces were output by the actor network with precision to two decimal places and reserve as the chromosomes. Then the range of actuator output was defined as 20.00 N < f < 20.00 N. Finally, the merits and demerits of − individuals were evaluated by RMS value. Because this study is based on simulation, the RMS value can be obtained by MATLAB. The searching process of the strategy unit is shown in Figure6. The various evaluation values of the initial generation reflect the variance of the individual. As the number of generations increases, the red points in the strategy unit gradually gather together, although some red points become irregular due to mutation. In the final stage, all red points were assembled in similar values until the algorithm converges, which means that the search is over and the optimal force has been found.

3.1.3. Datasets Update To further improve the correction accuracy of the proposed algorithm, the datasets were updated circularly. In the beginning, the parameters of the actor network and the strategy unit were randomly Sensors 2020, 20, 6403 8 of 16

Figure 5. The working procedure of the DLCA.

3. Results

3.1. Determination of Network Parameters and Datasets Update

3.1.1. The Parameters Setting of the Actor Network To construct and test the actor network, the 5000 collected data were segregated into a training set, a validation set, and a testing set in a manner consistent with a 3:1:1 ratio [39]. All three sets of data have different roles. The training set is used to identify parameters and constructs the actor network, the validation set is applied to evaluate the actor network, and the testing set is utilized to judge the final performance of the actor network. Detailed parameter settings of the actor network are listed in Table 2.

Sensors 2020, 20, 6403 Table 2. The parameter setting of the actor network. 9 of 15

FC1 FC2 FC3 FC4 FC5 Learning Activation Functions Optimizer initialized.Nodes Nodes The actor Nodes network wasNodes trained Nodes in advance according to the datasets obtained by MATLAB.Rate After65 training,145 the actor200 network 120 output the21 estimated force ReLU according to the initial Adam mirror shape. 0.001 Then, the evolutionary strategy algorithm was used in the strategy unit, which makes a pass through all of 3.1.2.the available The Configuration individuals of in the every Strategy iteration Unit and predicts the RMS value of each individual. It will kill the weaker individuals until all of them converge to one place. Finally, the optimal correction force was appliedTo toimprove the mirror. the correction The corrected accuracy mirror of shape the wasactive judged optics whether system, it canthe meetevolutionary the requirements, strategy algorithmif not, the correctionmentioned process above willwas beapplied executed to reduce again; otherwise,the search thetime. correction The 21 willcorrection be ﬁnished. forces Atwere the outputend of theby the correction, actor network a set of with random precision correction to tw forceso decimal were inputtedplaces and to reserve disturb theas the mirror chromosomes. shape after Thencorrection the range for the of nextactuator training. output The was above defined description as −20.00 was N implemented< < 20.00 N. in Finally, MATLAB. the Speciﬁcally,merits and demeritsthe data duringof individuals each correction, were evaluated including by RMS the mirror value. shape Because before this correction,study is based the optimalon simulation, correction the RMSforce, value and the can mirror be obtained shape afterby MATLAB. correction, were updated to the datasets. The searching process of the strategy unit is shown in Figure 6.

Sensors 2020, 20, 6403 9 of 16

randomly initialized. The actor network was trained in advance according to the datasets obtained by MATLAB. After training, the actor network output the estimated force according to the initial mirror shape. Then, the evolutionary strategy algorithm was used in the strategy unit, which makes a pass through all of the available individuals in every iteration and predicts the RMS value of each individual. It will kill the weaker individuals until all of them converge to one place. Finally, the optimal correction force was applied to the mirror. The corrected mirror shape was judged whether it can meet the requirements, if not, the correction process will be executed again; otherwise, the correction will be finished. At the end of the correction, a set of random correction forces were inputted to disturb the mirror shape after correction for the next training. The above description was implemented in MATLAB. Specifically, the data during each correction, including the mirror shape before correction, the optimalFigure correction 6. TheThe searchforce, search anprocess processd the of mirror the strategy shape unit.after correction, were updated to the datasets. 3.2. ComparisonThe various of evaluation Different Correction values of Algorithm the initial generation reflect the variance of the individual. As the number of generations increases, the red points in the strategy unit gradually gather together, 3.2.In Comparison this study, of Different a random Correction disturbance, Algorithm which follows the normal distribution of µ = 0 and σ = 1, although some red points become irregular due to mutation. In the final stage, all red points were wasIn addedthis study, to the a stirandomffness matrixdisturbance, to simulate which the follows disturbances the normal in the distribution real system of by μ MATLAB.= 0 and σ = To 1, validatewas assembled in similar values until the algorithm converges, which means that the search is over and theadded correction to the stiffness capability matrix and feasibilityto simulate of the the distur proposedbances algorithm, in the real system five diff byerent MATLAB. types of To initial validate mirror the optimal force has been found. shapes,the correction including capability RMS of and 0.26 feasibilityλ, 0.44λ, 0.68of theλ, 0.84proposedλ, and algorithm, 1.07λ, were five selected. different Simultaneously, types of initial mirror the most widelyshapes, used including least square RMS of algorithm 0.26λ, 0.44λ was, 0.68 chosenλ, 0.84λ, as and a comparison1.07λ, were selected. to carrying Simultaneously, out the correction the most force 3.1.3. Datasets Update underwidely the used same least condition, square algorithm the specific was calculation chosen as ofa comparison the least square to carrying algorithm out the refers correction to [11]. force underToFigure thefurther 7same shows improvecondition, the correction the the correctionspecific results calculation ofaccuracy the least of the squareof theleast algorithmproposed square algorithm (LSA)algorithm, and refers the the to DLCA. [11].datasets From were left updatedto rightFigure are circularly. the 7 shows initial In mirrorthe the correction beginning, shape beforeresults the parameters correction, of the least the ofsquare the mirror actor algorithm shape network is (LSA) corrected and and the the by strategy theDLCA. LSA, unit From and were the mirrorleft toshape right are is corrected the initial by mirror the DLCA. shape before correction, the mirror shape is corrected by the LSA, and the mirror shape is corrected by the DLCA.

(a)

Figure 7. Cont.

(b) Sensors 2020, 20, 6403 9 of 16

3.2. Comparison of Different Correction Algorithm In this study, a random disturbance, which follows the normal distribution of μ = 0 and σ = 1, was added to the stiffness matrix to simulate the disturbances in the real system by MATLAB. To validate the correction capability and feasibility of the proposed algorithm, five different types of initial mirror shapes, including RMS of 0.26λ, 0.44λ, 0.68λ, 0.84λ, and 1.07λ, were selected. Simultaneously, the most widely used least square algorithm was chosen as a comparison to carrying out the correction force under the same condition, the specific calculation of the least square algorithm refers to [11]. Figure 7 shows the correction results of the least square algorithm (LSA) and the DLCA. From left to right are the initial mirror shape before correction, the mirror shape is corrected by the LSA, and the mirror shape is corrected by the DLCA.

Sensors 2020, 20, 6403 10 of 15 (a)

Sensors 2020, 20, 6403 10 of 16 (b)

(c)

(d)

(e)

FigureFigure 7. 7.Mirror Mirror shape before before and and after after correction. correction. (a) The (a) initial The initial RMS (Root RMS Mean (Root Square) Mean of Square) the mirror of the mirroris 0.26 isλ;0.26 (b) λThe;(b initial) The initialRMS of RMS the mirror of the is mirror 0.44λ; is(c 0.44) Theλ ;(initialc) The RMS initial of the RMS mirror of the is 0.68 mirrorλ; (d is) The 0.68 λ; (d)initial Theinitial RMS of RMS the ofmirror the mirroris 0.84λ is; (e 0.84) Theλ;( initiale) The RMS initial of the RMS mirror of the is mirror1.07λ. is 1.07λ.

Apparently, the deformation of the mirror was modified by the two algorithms after correction. Notably, the correction performance of the DLCA is always better than the LSA regardless of the initial mirror shape. The detailed correction results of the LSA and the DLCA are shown in Table 3.

Sensors 2020, 20, 6403 11 of 15

Apparently, the deformation of the mirror was modiﬁed by the two algorithms after correction. Notably, the correction performance of the DLCA is always better than the LSA regardless of the initial mirror shape. The detailed correction results of the LSA and the DLCA are shown in Table3.

Table 3. Comparison of correction results for diﬀerent algorithms.

Before After Correction Method. Correction Correction Times LSA 0.26 0.05 3 DLCA 0.26 0.01 1 LSA 0.44 0.05 4 DLCA 0.44 0.01 1 LSA 0.68 0.06 4 DLCA 0.68 0.01 1 LSA 0.84 0.05 5 DLCA 0.84 0.02 2 LSA 1.07 0.06 6 DLCA 1.07 0.02 2

In the first experiment, the initial RMS of the mirror was 0.26λ. After correction with the LSA, the RMS of the mirror was reduced to 0.05λ, and the number of corrections was three. Compared with LSA, the DLCA reduces the RMS from 0.26λ to 0.01λ, which correction times are one. In the second experiment, the initial RMS of the mirror was 0.44λ. After correction with the LSA, the RMS of the mirror was reduced to 0.05λ, and the number of corrections is four. Compared with LSA, the DLCA reduces the RMS from 0.44λ to 0.01λ, which correction times are still one. In the third experiment, the initial RMS of the mirror was 0.68λ. After correction with the LSA, the RMS of the mirror was reduced to 0.06λ, and the number of corrections was still four. Compared with LSA, the DLCA reduced the RMS from 0.68λ to 0.01λ, whose correction times are one. To further illustrate the feasibility of the proposed algorithm, the initial RMS of the mirror was set worse. Therefore, in the fourth and fifth experiments, the initial RMS of the mirror was 0.84λ, and 1.07λ. After correction with the LSA, the RMS of the mirror was reduced to 0.05λ and 0.06λ. However, the number of corrections increased to 5 and 6. Compared with LSA, the DLCA reduced the RMS from 0.84λ to 0.02λ and 1.07λ to 0.02λ. The correction times were all two. According to the above results, we can find that the LSA takes about 3–5 times on average to complete the correction, and the RMS value after the correction is about 0.05λ. While the DLCA only needs 1–2 times to complete the correction, and the RMS value after the correction is about 0.01λ. Therefore, we can conclude that DLCA has better correction performance and can effectively reduce the RMS of the mirror. Compared with LSA, DLCA only needs fewer correction times and has higher correction accuracy. Furthermore, to illustrate the superiority of the proposed algorithms, we compared the correction performance of the LSA and DLCA on Zernike mode. Since the four items of Zernike polynomial, translation, tilt, and defocus, can be corrected by other methods, the four items were not selected. Meanwhile, considering the poor ability of active optics to correct higher-order aberrations, only the 5th–14th items are selected. As mentioned above, five types of initial mirror shapes were also selected, the correction results of the LSA and the DLCA in Zernike mode are shown in Figure8. As can be seen from Figure8, compared with the LSA, the low order Zernike aberrations, especially the astigmatisms, were significantly reduced after correction when adopting the DLCA. Additionally, the reduction of high order Zernike aberrations is relatively weak, which shows that the correction capability for high order Zernike aberrations is poorer than the low order Zernike aberrations. Based on the analysis above, we can find that the DLCA has a stronger ability to correct aberrations than LSA. Sensors 2020, 20, 6403 12 of 15

Sensors 2020, 20, 6403 12 of 16

(a) (b)

(e)

Figure 8. The correction results of LSA and DLCA in Zernike mode. (a) The initial RMS of the mirror is Figure0.26λ;( 8.b )The The correction initial RMS results of the of mirror LSA and is 0.44 DLCAλ;(c) in The Zernike initial RMSmode. of (a the) The mirror initial is 0.68RMSλ;( ofd the) The mirror initial isRMS 0.26λ of; ( theb) The mirror initial is 0.84 RMSλ;( ofe) the The mirror initial RMSis 0.44 ofλ; the (c) mirrorThe initial is 1.07 RMSλ. of the mirror is 0.68λ; (d) The initial RMS of the mirror is 0.84λ; (e) The initial RMS of the mirror is 1.07λ. 4. Discussion AsThe can support be seen system from of Figure the active 8, opticscompared platform with consists the LSA, of 24the axial low supports order Zernike and 3 lateral aberrations, supports especiallyin the experiments. the astigmatisms, Under this were condition, significantly the proposed reduced algorithm after correction has excellent when correction adopting performance.the DLCA. Additionally,However, this the does reduction not affect of thehigh application order Zernike of the aberrations proposed algorithm is relatively in otherweak, conditions, which shows nor that does theit a correctionffect the universality capability for of thehigh proposed order Zernike algorithm. aberrations In other is words, poorer regardless than the low of the order number Zernike and aberrations.distribution Based of actuators, on the analysis the proposed above, algorithm we can find in this that study the DLCA can also has calculate a stronger the ability correction to correct force. aberrationsTo simulate than LSA. the real system, a random disturbance is generated and added to the stiffness matrix by MATLAB in this paper. Therefore, all the datasets used for training contain the disturbance. For the disturbance of other factors, the algorithm proposed in this paper can also be applied, only if the Sensors 2020, 20, 6403 13 of 15 corresponding disturbance is added to the datasets. In other words, the datasets used for training must contain the corresponding disturbance, so that the deep neural network can fit the active optics system with disturbances. For the real system, this is easier to implement because of the input and output data collected by the real system that already contains various disturbances, such as thermal distortion, and gravity deformation, especially the error in structure and actuator control. In addition, to adequately illustrate the advantages of the proposed algorithm in correction performance, the comparison of the least square algorithm and the algorithm proposed in this paper is carried out. The data of Table3 shows that the correction performance of the proposed algorithm is always better than the LSA regardless of the initial mirror shape, which indicates that the proposed algorithm has a better correction capability that deals with the mirror shape with the disturbance. Therefore, we conclude that the proposed algorithm has better adaptability and correction capability than the traditional algorithm in the case of the disturbance. From the above results, it is known that the proposed algorithm can correct wavefront aberrations more effectively than the traditional algorithm. However, there are still many high-order aberrations that cannot be corrected, which indicates that the proposed algorithm has a good correction performance of the low-order aberrations, but the ability to correct high-order aberrations is relatively weak. Although the mirror shape may have been corrected well, there are still high-order aberrations. In this case, only adaptive optics can be used for correction with its advantage of correcting the high-order aberrations.

5. Conclusions A deep learning correction algorithm is proposed in this paper for the correction of wavefront aberrations, which is based on the data rather than the mirror’s deformation. Therefore, the proposed algorithm can deal with the active optics system with disturbances better. Five different types of initial mirror shapes, including 0.26λ, 0.44λ, 0.68λ, 0.84λ, and 1.07λ, are utilized to validate the proposed algorithm. The RMS of mirror shape is significantly reduced and the low-order aberrations are also reduced after correction. Additionally, compared with the least squares algorithm, the performance of correct wavefront aberrations is overall improved, which indicates that the algorithm has a better correction capability. The results also provide us with some inspirations for future work. Firstly, the evolutionary strategy algorithm is utilized to reduce the search time, but the search process still takes much time, especially when the number of actuators further increases. We will continue to focus on how to reduce the search time of the strategy unit. Secondly, to improve the speed of practical application, we will continue to try to design a new input way so that the correction force can be calculated directly from the figure of the mirror shape. Thirdly, we will try to apply the proposed algorithm to large modern telescopes in the future study. The contributions of this paper are as follows: (1) A new correction algorithm based on deep learning is proposed, which can improve the correction capability of the mirror; (2) Deep learning is introduced into the active optics system and provides a new train of thought for the further research of how to improve the correction capability of the mirror.

Author Contributions: W.L. and C.K. contributed equally to this work. Conceptualization, W.L. and C.K.; methodology, W.L. and C.K.; investigation, H.G.; software, S.H.; data curation, J.Z.; writing—original draft preparation, W.L.; writing—review and editing, X.Z. and J.L. All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by the National Natural Science Foundation of China (12003067) and the Foundation of Shanghai Key Laboratory for Astrophysics (SKLA1901). Conﬂicts of Interest: The authors declare no conﬂict of interest. Sensors 2020, 20, 6403 14 of 15

References

1. Schipani, P.; Noethe, L.; Arcidiacono, C.; Argomedo, J.; Dall’Ora, M.; D’Orsi, S.; Farinato, J.; Magrin, D.; Marty, L.; Ragazzoni, R.; et al. Removing static aberrations from the active optics system of a wide-field telescope. J. Opt. Soc. Am. A 2012, 29, 1359–1366. [CrossRef][PubMed] 2. Schipani, P.; Noethe, L.; Magrin, D.; Kuijken, K.; Arcidiacono, C.; Argomedo, J.; Capaccioli, M.; Dall’Ora, M.; D’Orsi, S.; Farinato, J.; et al. Active optics system of the VLT Survey Telescope. Appl. Opt. 2016, 55, 1573–1583. [CrossRef][PubMed] 3. Cross, N.J.G.; Collins, R.S.; Mann, R.G.; Read, M.A.; Sutorius, E.T.W.; Blake, R.P.; Holliman, M.; Hambly, N.; Emerson, J.P.; Lawrence, A.; et al. The VISTA Science Archive. Astron. Astrophys. 2012, 548, A119. [CrossRef] 4. Thomas, S.J.; Xin, B.; Tsai, T.-W.; Contaxis, C.; Claver, C.; Lotz, P.; Neill, D. The LSST real-time active optics system. In Proceedings of the 5th Adaptive Optics for Extremely Large Telescopes (AO4ELT 2017), Tenerife, Spain, 25–30 June 2017. [CrossRef] 5. Wang, H.; Liang, M.; Yao, D.; Zuo, Y.; Zheng, X.; Yang, J. Study on the application of the free-vibration modes of an annular mirror in the active optics system. J. Astron. Telesc. Instrum. Syst. 2020, 6.[CrossRef] 6. Hosoya, N.; Niikura, T.; Hashimura, S.; Kajiwara, I.; Giorgio-Serchi, F. Axial force measurement of the bolt/nut assemblies based on the bending mode shape frequency of the protruding thread part using ultrasonic modal analysis. J. Int. Meas. Confed. 2020, 162, 107914. [CrossRef] 7. Dolkens, D.; Van Marrewijk, G.; Kuiper, H. Active correction system of a deployable telescope for earth observation. In Proceedings of the International Conference on Space Optics (ICSO 2018), Chania, Greece, 9–12 October 2018. [CrossRef] 8. Chen, F.; Zhang, J.; Wu, X.; Sun, J.; Cong, J. Deformation of thin primary mirror fitted with its vibration mode. Infrared Laser Eng. 2011, 40, 2238–2243. 9. Zhu, Y.; Chen, T.; Wang, J.-L.; Li, H.-Z.; Wu, X.-X. Active correction of 1.23 m SiC mirror using bending mode. Opt. Precis. Eng. 2017, 25, 2551–2563. [CrossRef] 10. Han, Y.; Fan, B.; Li, C.; Liu, H. Analysis of surface error correction capability of 1.2m active support system. In Proceedings of the 8th International Symposium on Advanced Optical Manufacturing and Testing Technology (AOMATT2016), Suzhou, China, 26–29 April 2016; p. 96820Q. [CrossRef] 11. Dai, X.; Xian, H.; Tang, J.; Zhang, Y.Active correction experiment on a 12 m thin primary mirror. J. Opt. Technol. 2019, 86, 341–349. [CrossRef] 12. Zhou, P.; Zhang, D.; Liu, G.; Yan, C. Development of space active optics for a whiffletree supported mirror. Appl. Opt. 2019, 58, 5740–5747. [CrossRef] 13. Schwaer, C.; Sinn, A.; Schitter, G. Mechatronic approach towards lightweight mirrors with active optics for telescope systems. In Proceedings of the 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS), Vienna, Austria, 4–6 September 2019; pp. 7–12. [CrossRef] 14. Ashraf, I.; Hur, S.; Park, S.; Park, Y. DeepLocate: Smartphone Based Indoor Localization with a Deep Neural Network Ensemble Classifier. Sensors 2019, 20, 133. [CrossRef] 15. Cho, H.; Lee, H. Biomedical named entity recognition using deep neural networks with contextual information. BMC Bioinform. 2019, 20, 1–11. [CrossRef][PubMed] 16. Guo, H.; Xu, Y.; Li, Q.; Du, S.; He, D.; Wang, Q.; Huang, Y. Improved Machine Learning Approach for Wavefront Sensing. Sensors 2019, 19, 3533. [CrossRef][PubMed] 17. Hegde, R.S. Accelerating optics design optimizations with deep learning. Opt. Eng. 2019, 58, 58. [CrossRef] 18. Gómez, S.L.S.; González-Gutiérrez, C.; Riesgo, F.G.; Lasheras, F.S.; Lasheras, F.S.; Rodríguez, J.D.S. Convolutional Neural Networks Approach for Solar Reconstruction in SCAO Configurations. Sensors 2019, 19, 2233. [CrossRef][PubMed] 19. Xu, Z.; Yang, P.; Ke, H.; Xu, B.; Li, H. Deep learning control model for adaptive optics systems. Appl. Opt. 2019, 58, 1998–2009. [CrossRef] 20. Xu, Y.; Zhao, Q.; Li, J.; Jiao, C. Simulation Analysis of Large-Aperture Standard Planar Mirror Based on Active Correction. Available online: https://kns.cnki.net/kcms/detail/detail. aspx?dbcode=CJFD&dbname=CJFDLAST2019&filename=JGHW201902018&v=pm7dIz7lJJR5uc% 25mmd2BWJQ48EJCtMf6nH7TRua7n4bUgG4ircbxWsFb1QjgWPqeI1RfM (accessed on 9 November 2020). (In Chinese) Sensors 2020, 20, 6403 15 of 15

21. Li, H.; Zhang, J.; Zhang, Z.; Wang, H.; Wang, M. Correction experiment of 620 m thin mirror active optics telescope. Infrared Laser Eng. 2014, 43, 166–172. 22. Raimondi, L.; Manfredda, M.; Mahne, N.; Cocco, D.; Capotondi, F.; Pedersoli, E.; Kiskinova, M.; Zangrando, M. Kirkpatrick–Baez active optics system at FERMI: System performance analysis. J. Synchrotron Radiat. 2019, 26, 1462–1472. [CrossRef] 23. Spiga, D.; Barbera, M.; Basso, S.; Civitani, M.; Collura, A.; Dell’Agostino, S.; Lo Cicero, U.; Lullo, G.; Pelliciari, C.; Riva, M.; et al. Active shape correction of a thin glass/plastic X-ray mirror. In Proceedings of the SPIE Optical Engineering + Applications: Adaptive X-ray Optics III, San Diego, CA, USA, 17–21 August 2014; p. 92080A. [CrossRef] 24. Lecun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [CrossRef] 25. Duchanoy, C.A.; Moreno-Armendáriz, M.A.; Moreno-Torres, J.C.; Cruz-Villar, C.A. A Deep Neural Network Based Model for a Kind of Magnetorheological Dampers. Sensors 2019, 19, 1333. [CrossRef] 26. Guo, L.-X.; Dao, D.-N. A new control method based on fuzzy controller, time delay estimation, deep learning, and non-dominated sorting genetic algorithm-III for powertrain mount system. J. Vib. Control. 2019, 26, 1187–1198. [CrossRef] 27. Yoon, K.; Kim, D.Y.; Yoon, Y.-C.; Jeon, M. Data Association for Multi-Object Tracking via Deep Neural Networks. Sensors 2019, 19, 559. [CrossRef][PubMed] 28. Kingma, D.P.; Ba, J.L. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, CA, USA, 7–9 May 2015. 29. Lopes, F.F.; Ferreira, J.C.; Fernandes, M.A.C. Parallel Implementation on FPGA of Support Vector Machines Using Stochastic Gradient Descent. Electronics 2019, 8, 631. [CrossRef] 30. Guo, X.; Zhang, J.; Tie, L.; Luo, M. HS-SA-Based Precise Modeling of the Aircraft Fuel Center of Gravity Using Sensors Data. Sensors 2019, 19, 2457. [CrossRef][PubMed] 31. Guo, J.; Yuan, W.; Dang, X.; Alam, M.S. Cable force optimization of a curved cable-stayed bridge with combined simulated annealing method and cubic B-Spline interpolation curves. Eng. Struct. 2019, 201, 109813. [CrossRef] 32. Feng, Y.; Zhou, M.; Tian, G.; Li, Z.; Zhang, Z.; Zhang, Q.; Tan, J. Target Disassembly Sequencing and Scheme Evaluation for CNC Machine Tools Using Improved Multiobjective Ant Colony Algorithm and Fuzzy Integral. IEEE Trans. Syst. Man Cybern. Syst. 2018, 49, 2438–2451. [CrossRef] 33. Sun, G.-C.; Xiang, J.; Xing, M.; Yang, J.; Guo, L. A Channel Phase Error Correction Method Based on Joint Quality Function of GF-3 SAR Dual-Channel Images. Sensors 2018, 18, 3131. [CrossRef] 34. Yan, C.; Li, M.X.; Liu, W. Application of Improved Genetic Algorithm in Function Optimization. J. Inf. Sci. Eng. 2019, 35, 1299–1309. [CrossRef] 35. Ningombam, D.D.; Shin, S. Optimal Resource Management and Binary Power Control in Network-Assisted D2D Communications for Higher Frequency Reuse Factor. Sensors 2019, 19, 251. [CrossRef] 36. Larik, A.; Haider, S. A framework based on evolutionary algorithm for strategy optimization in robot soccer. Soft Comput. 2018, 23, 7287–7302. [CrossRef] 37. Qiu, J.; Liu, M.; Zhang, L.; Li, W.; Cheng, F. A multi-level knee point based multi-objective evolutionary algorithm for AUC maximization. Memetic Comput. 2019, 11, 285–296. [CrossRef] 38. Yang, F.; Ren, H.; Hu, Z. Maximum Likelihood Estimation for Three-Parameter Weibull Distribution Using Evolutionary Strategy. Math. Probl. Eng. 2019, 2019, 1–8. [CrossRef] 39. Yadav, S.; Shukla, S. Analysis of k-Fold Cross-Validation over Hold-Out Validation on Colossal Datasets for Quality Classiﬁcation. In Proceedings of the 2016 IEEE 6th International conference on advanced computing (IACC), Bhimavaram, India, 27–28 February 2016; pp. 78–83. [CrossRef]

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional aﬃliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).