A Knowledge-Informed Simplex Search Method Based on Historical Quasi-Gradient Estimations and Its Application on Quality Control of Medium Voltage Insulators

processes

Article A Knowledge-Informed Simplex Search Method Based on Historical Quasi-Gradient Estimations and Its Application on Quality Control of Medium Voltage Insulators

Xiangsong Kong * and Dongbin Zheng

School of Electrical Engineering and Automation, Xiamen University of Technology, Xiamen 361024, China; [email protected] * Correspondence: [email protected]

Abstract: Quality control is of great significance for the economical manufacturing and reliable application of medium voltage insulators. With the increasingly stringent quality control requirement, traditional quality control methods in this field face a growing challenge on their efficiency. Therefore, this study aims to achieve quality specifications by optimizing process conditions with the least costs. Thus, a knowledge-informed simplex search method was proposed based on an idea of knowledge- informed optimization to enhance the optimization efficiency. Firstly, a new mathematical quantity, quasi-gradient estimation, was generated following a reconstruction of the simplex search from the essence and the development history of the method. Based on this quantity, the gradient-free method possessed the same gradient property and unified form as the gradient-based methods. Secondly, an implementation of the knowledge-informed simplex search method based on historical quasi- gradient estimations (short for GK-SS) was constructed. The GK-SS-based quality control method utilized the historical quasi-gradient estimations for each simplex generated during the optimization Citation: Kong, X.; Zheng, D. A process to improve the method’s search directions’ accuracy in a statistical sense. Finally, this method Knowledge-Informed Simplex Search Method Based on Historical was applied to the weight control of a kind of post insulator. The experimental simulation results Quasi-Gradient Estimations and Its showed that the method is effective and efficient in the quality control of medium voltage insulators. Application on Quality Control of Medium Voltage Insulators. Processes Keywords: quality control; medium voltage insulator; knowledge-informed; simplex search method; 2021, 9, 770. https://doi.org/ quasi-gradient estimations 10.3390/pr9050770

Academic Editor: Xi Chen 1. Introduction Received: 1 December 2020 Medium voltage insulators, which are widely used in electrical switches, electrical Accepted: 13 April 2021 control devices, and other electrical equipment [1,2], are manufactured by the epoxy resin Published: 28 April 2021 automatic pressure gelation process (APG) [3]. As the APG process parameters significantly impact product quality, insulators’ quality control is mainly achieved via the tuning of Publisher’s Note: MDPI stays neutral process parameters [4,5]. However, the APG process is a typical batch process with high with regard to jurisdictional claims in complexity and nonlinearity [6]. The challenging task, developing quality control methods published maps and institutional affil- with high efficiency, is at high priority. iations. Traditional quality control methods for batch processes become increasingly inefficient to meet the stringent quality control requirement. The trial-and-error and the design of experiments (DOE) are suboptimal, time-consuming, and experience-dependent [7]. The model-based optimization (MBO) methods rely on reliable quality models, which cannot Copyright: © 2021 by the authors. avoid model mismatch and is highly inefficient on model building. To address the above Licensee MDPI, Basel, Switzerland. challenges, a systematic model-free optimization (MFO) framework for a type of batch This article is an open access article processes with a short cycle time and a low operational cost was proposed [7–9]. In the distributed under the terms and MFO framework, direct experiments are utilized to substitute the role of the model in conditions of the Creative Commons Attribution (CC BY) license (https:// MBO. Hence, it can retain the high-efficiency optimization characteristics of MBO and creativecommons.org/licenses/by/ simultaneously avoids high modeling costs. To reduce the experimental costs of MFO 4.0/). further, Zhao et al. proposed an iterative modeling and the trust-region optimization

Processes 2021, 9, 770. https://doi.org/10.3390/pr9050770 https://www.mdpi.com/journal/processes Processes 2021, 9, 770 2 of 24

method (IMTO) by integrating the advantages of both the MFO and the MBO together [10]. Lu et al. proposed a model-free quality optimization method based on the natural gradient for the injection molding processes [11]. These methods significantly improved this kind of batch processes’ quality control efficiency. However, as the operational cost and the cycle time for a single batch of the APG are relatively higher, more efficient method is needed. With the current emerging trend on data-driven technology, data-driven control or optimization is becoming a research hotspot. Spall and Dong et al. tried to revise the traditional model-free algorithms to enhance efficiency [12,13]. Hou et al. focused on the development of the data-driven strategy to improve the efficiency of model-free control (MFC) [14–16]. Xing et al. had proposed a series of knowledge-based algorithms and learning-guided algorithms [17,18]. In all the above works, knowledge generated during the optimization or obtained prior is utilized to promote method efficiency. Based on a knowledge-informed idea, Kong et al. has proposed different kinds of knowledge-informed Simultaneous Perturbation Stochastic Approximation (SPSA) methods and applied them to medium voltage insulators’ quality control [4,5,19]. However, the quality control of medium voltage insulator is mostly a low-dimensional problem, while SPSA, as a gradient-based method, is more suitable for high-dimensional optimization problems. Another kind of gradient-free method, the simplex search method, which is espe- cially effective for low-dimensional optimization, was concerned. As a popular algorithm, its efficiency has gained much attention. Researchers have tried to investigate it further and modify it in different forms [20–25]. These modifications, however, do not suitable for the scenarios of MFO-based quality control. By now, there are no modifications in the simplex search method from the perspective of using iterative optimization process knowledge. For the SPSA-based MFO, different kinds of knowledge generated during the optimization process have been extracted and utilized to improve its optimization efficiency. In this study, it was expected that the same knowledge-informed idea might be employed for the simplex search method. The development and the characteristics of the traditional simplex search method were detailed and analyzed. On the basis, a revised simplex search method, knowledge-informed simplex search based on historical gradient approximations (GK-SS), was proposed. As a method based on an idea of knowledge-informed optimization, the GK-SS integrates a kind of iteration knowledge, the quasi-gradient estimations, generated during the optimization process to improve the efficiency of quality control for a type of batch process with relatively high operational costs. The remainder of this paper is organized as follows. Section2 illustrates the quality control problem of medium voltage insulators. Section3 presents an overview of the knowledge-informed optimization strategy, followed by the simplex search method’s basic principles. The development history of the traditional simplex search was detailed based on the idea of a knowledge-informed optimization strategy, and a revised knowledge- informed simplex search method based on historical quasi-gradient estimations (GK-SS) was proposed and delineated. In Section4, the GK-SS method was further verified on the weight control of a kind of post insulator. A comprehensive discussion is carried out to assess GK-SS’s superiority over the SPSA and the traditional simplex search method based on the experimental results. Finally, conclusions were summarized.

2. Quality Control for Medium Voltage Insulators In our previous paper (Referred to [4]), we had defined and illustrated the problem of quality control for medium voltage insulators in detail. According to the previous definition, we mainly achieve the medium voltage insulators’ quality control in this study by adjusting the APG process’s parameters. Accordingly, the quality control of the insulators was transformed into searching for the optimal process parameters. The relationship between the quality response of medium voltage insulators and the process parameters is illustrated in Figure1. The mathematical expression of the quality model is defined as below:

T Q = f (x) + e, x = [x1, ··· xn] (1) Processes 2021, 9, x FOR PEER REVIEW 3 of 26

The relationship between the quality response of medium voltage insulators and the process parameters is illustrated in Figure 1. The mathematical expression of the quality model is defined as below: =+=() []T Qfxexx,,1 L xn (1) Processes 2021, 9, 770 3 of 24 where Q is the quality response of the quality index to be controlled, x is an n-dimen- ()= sional vector for the APG process parameters, and xi in1,L , is the ith process pa- rameterswhere Q ofis thex , qualityn is the response dimension of the of quality x , e index represents to be controlled, the processx isuncertainty, an n-dimensional while ( = ··· ) vectorf ()x foris the the internal APG process mechanism parameters, between and thex qui iality1, index, nandis the the correspondingith process parameters process of x, n is the dimension of x, e represents the process uncertainty, while f (x) is the internal parameters.mechanism As between analyzed the previously, quality index the explicit and the expression corresponding of the processquality model parameters. is usually As unavailable.analyzed previously, the explicit expression of the quality model is usually unavailable.

Figure 1. Relationship between the quality response of medium voltage insulators and the Figure 1. Relationship between the quality response of medium voltage insulators and the process conditions.process conditions. In this study, quality control of ﬁnding the optimal process conditions can be summa- In this study, quality control of finding the optimal process conditions can be sum- rized as below: marized as below: min QE = |Q − Qt| x s.t. RL ≤ x ≤ −RH, i = 1, 2, ··· , n; (2) min iQQQEt=i i x T x = [x1, ··· xn] . LH≤≤ = st.. Riii x R , i 1,2,L , n ; (2) where QE, as the objective function, is the absolute quality error between the actual quality L TH response Q and the quality target Q=t. []R and R are the lower and upper bounds of xx1i,.L xn i xi, respectively. According to Equation (2), quality control of medium voltage insulators is essentially where QE , as the objective function, is the absolute quality error between the actual qual- a nonlinear optimization problem with bounded constraints.LH The target of quality control ityis toresponse attain the Q optimal and the process quality parameters target Q withint . Rii qualityand R speciﬁcations are the lower at the and least upper costs.

boundsConsidering of xi ,that respectively. the APG process’s quality model is usually unavailable, the model- freeAccording optimization to Equation framework (2), (MFO) quality is control supposed of medium to be suitable voltage for insulators the quality is essentially control of asuch nonlinear batch optimization processes with problem run-to-run with bounded repetitive constraints. characteristics. The target The schematic of quality diagram control isof to APG’s attain qualitythe optimal control process is illustrated parameters in Figurewithin2 quality. In the specifications MFO framework, at the theleast quality costs. Consideringevaluation is that conducted the APG online, process’s which quality substitutes model the is qualityusually model’s unavailable, role in the the model-free MBO. With optimizationthis closed-loop framework and online (MFO) quality is supposed control strategy,to be suitable medium for the voltage quality insulators’ control of quality such batchspeciﬁcations processes could with be run-to-run achieved iteratively.repetitive characteristics. The detailed procedure The schematic of the MFOdiagram can of be APG’sreferred quality to [7]. control This methodology is illustrated initially in Figure sprung 2. In the from MFO considering framework, the the quality quality control evalu- for a type of batch process with a short cycle time and a low operation cost. However, the ation is conducted online, which substitutes the quality model’s role in the MBO. With application scenario is slightly different in quality control of the medium voltage insulators, this closed-loop and online quality control strategy, medium voltage insulators’ quality which has a relatively higher price for each batch. As discussed in [4], the APG’s quality specifications could be achieved iteratively. The detailed procedure of the MFO can be control is sensitive to the iteration number. Thus, the iteration costs on quality control referred to [7]. This methodology initially sprung from considering the quality control for should be minimized to improve efﬁciency. In this study, the simplex search method has a type of batch process with a short cycle time and a low operation cost. However, the been revised to utilize historical iteration knowledge from an idea of knowledge-informed application scenario is slightly different in quality control of the medium voltage insula- optimization to achieve the above goal. tors, which has a relatively higher price for each batch. As discussed in [4], the APG’s

Processes 2021, 9, x FOR PEER REVIEW 4 of 26

quality control is sensitive to the iteration number. Thus, the iteration costs on quality control should be minimized to improve efficiency. In this study, the simplex search Processes 2021, 9, 770 4 of 24 method has been revised to utilize historical iteration knowledge from an idea of knowledge-informed optimization to achieve the above goal.

FigureFigure 2.2. QualityQuality controlcontrol schematic schematic diagram diagram for for medium medium voltage voltage insulator insulator manufacturing manufacturing process. process. [4]. [4]. 3. Knowledge-Informed Simplex Search Methodology 3.1.3. Knowledge-Informed Knowledge-Informed Optimization Simplex Search Strategy Methodology 3.1. Knowledge-InformedSince the operational Optimization cost of the Strategy APG is relatively high, the number of batches on qualitySince control the ofoperational medium voltage cost of insulators the APG hasis re alatively significant high, impact the number on the economy of batches of the on qualityquality control control process. of medium The rapidity,voltage insulators therefore, is has of paramounta significant importance impact on for the the economy quality control of medium voltage insulators. It is necessary to reduce the number of iterative of the quality control process. The rapidity, therefore, is of paramount importance for experiments in the quality optimization process as much as possible, thus reducing the the quality control of medium voltage insulators. It is necessary to reduce the number cost of quality control. Even though in the optimization framework, as shown in Figure2, theof iterative MFO method experiments has excellent in the performance quality optimi on qualityzation controlprocess of as a much type of as batch possible, processes thus withreducing lower the operational cost of quality cost. As control. the scenario Even is though slightly in different the optimization for the APG, framework, the traditional as MFO,shown however, in Figure faces 2, the the MFO relatively method high has optimization excellent priceperformance challenge. on quality control of a typeUp of to batch the present, processes there with is nolower method operatio superiornal cost. to theAs MFOthe scenario that exists is slightly to address dif- thisferent challenge. for the APG, Considering the traditional that the MFO, MFO stillhowever, has its faces advantages the relatively in the medium high optimiza- voltage insulators’tion price challenge. quality control, a possible way to promote quality control efficiency is to revise the conventionalUp to the present, MFO method. there is Underno method the framework superior to of the MFO, MFO the that optimization exists to methodaddress isthis the challenge. key. Consequently, Considering it isthat critical the MFO to further still has enhance its advantages the optimization in the medium method’s volt- ef- ficiencyage insulators’ under thequality existing control, optimization a possible framework. way to promote During quality the optimization, control efficiency a series is ofto iterativerevise the process conventional information, MFO which method. contains Under process the framework knowledge, of MFO, will be the generated optimi- dynamically.zation method In is traditional the key. Consequently, optimization methods, it is critical most to offurther this knowledge enhance the is discardedoptimiza- ortion not method’s used effectively. efficiency If under this knowledge the existing can optimization be extracted andframework. utilized effectively,During the itop- is possible to further enhance the optimization method’s efficiency. It is feasible to reduce the timization, a series of iterative process information, which contains process optimization number and improve the quality control efficiency for a type of batch process knowledge, will be generated dynamically. In traditional optimization methods, most with relatively high operation cost by mining the process knowledge generated during the optimization.of this knowledge Based is on discarded the knowledge-informed or not used effectively. idea, different If this knowledge-informed knowledge can be opti- ex- mizationtracted and strategies utilized for SPSAeffectively, had already it is possible been formulated. to further It was enhance verified the that optimization appreciable improvementmethod’s efficiency. could be It achieved. is feasible to reduce the optimization number and improve the qualityThe control simplex efficiency search method, for a type as aof gradient-free batch process optimization with relatively strategy, high is operation different fromcost by SPSA. mining Even the though process the knowledge simplex search generated method’s during principle the optimization. is quite different Based from on SPSA,the knowledge-informed the basic idea of the idea, knowledge-informed different knowledge-informed optimization strategy optimization for both strategies of them isfor similar. SPSA had Like already SPSA, a been certain formulated. amount of It process was verified knowledge that appreciable will be generated improvement during thecould iterative be achieved. process of the simplex search optimization. The simplex search takes each simplex,The whichsimplex consists search of method, n + 1 points, as a gradient-free as an iteration optimization point. It determines strategy, is its different current searchfrom SPSA. direction Even and though the step the size simplex only according search method’s to the current principle simplex is quite information different at from each iteration.SPSA, the The basic knowledge idea of ofthe the knowledge-in historical sequentialformed simplices, optimization such asstrategy simplex for sequences, both of centroids,them is similar. and historical Like SPSA, reflection a certain points, amount is completely of process discarded. knowledge However, will be the generated historical knowledge can be utilized to enhance the optimization method’s efficiency if appropriate during the iterative process of the simplex search optimization. The simplex search mechanisms could be built according to the characteristics of the simplex search. For the quality control of medium voltage insulators, this knowledge can be used to improve the simplex search method. Therefore, the knowledge-informed quality control strategy for the simplex-search-based MFO is promising. The framework of the knowledge-informed quality optimization via the simplex search method is shown in Figure3. Stem from the above-mentioned knowledge-informed idea, this section discusses a kind of knowledge- Processes 2021, 9, x FOR PEER REVIEW 5 of 26

takes each simplex, which consists of n + 1 points, as an iteration point. It determines its current search direction and the step size only according to the current simplex information at each iteration. The knowledge of the historical sequential simplices, such as simplex sequences, centroids, and historical reflection points, is completely discarded. However, the historical knowledge can be utilized to enhance the optimization method’s efficiency if appropriate mechanisms could be built according to the characteristics of the simplex search. For the quality control of medium voltage insulators, this knowledge can be used to improve the simplex search method. Therefore, the knowledge-informed quality control strategy for the simplex-search-based MFO Processes 2021, 9, 770 5 of 24 is promising. The framework of the knowledge-informed quality optimization via the simplex search method is shown in Figure 3. Stem from the above-mentioned knowledge-informed idea, this section discusses a kind of knowledge-informed sim- informedplex search simplex method search based method on historical based on qu historicalasi-gradient quasi-gradient estimations estimations in-depth. in-depth.A feasi- Able feasible implementation implementation mechanism mechanism of the of GK-SS the GK-SS strategy strategy was wasput putforward. forward.

FigureFigure 3.3. A framework of the knowledge-informed qua qualitylity control via the simplex search search method. method.

3.2.3.2. Basic Principles of Simplex Search Methodology As aa gradient-freegradient-free multivariate multivariate optimization optimization method, method, the the simplex simplex search search method method is a directis a direct search search method method for nonlinear for nonlinear optimization. optimization. This This method method is formulated is formulated based based on a concepton a concept of simplex, of simplex, which which is a geometric is a geometric object object that that is a convexis a convex hull hull of n of+ n 1 + points—not 1 points— lying in the same hyperspace—in n-dimensional Euclidean space Rn (n is the dimension not lying in the same hyperspace—in n-dimensional Euclidean space R n (n is the di- of the optimization problem understudy) [26]. This method’s origin can be traced back mension of the optimization problem understudy) [26]. This method’s origin can be to the idea of evolutionary operation (EVOP), which was proposed by Box in 1957 [27]. Consideringtraced back to how the EVOPidea of might evolutionary be automatic, operation Spendley (EVOP), et al. which proposed was proposed the original by ideaBox ofin the1957 sequential [27]. Considering simplex searchhow EVOP [28]. To might further be au improvetomatic, this Spendley method’s et efficiency,al. proposed Nelder the andoriginal Mead idea made of the simplexsequential search simplex method sear havech [28]. the capabilityTo further to improve adapt itself this tomethod’s the local landscapeefficiency, [Nelder29]. Due and to itsMead simplicity made andthe simplex effectiveness, search the method Nelder-Mead have the simplex capability search to methodadapt itself becomes to the widely local usedlandscape in industrial [29]. Du qualitye to its improvement simplicity and areas. effectiveness, As a result, the the Nelder-MeadNelder-Mead simplexsimplex search search (called method simplex becomes search widely by default) used in is theindustrial baseline quality method im- in thisprovement project. areas. The procedure As a result, of thethe simplexNelder-M searchead simplex method, search as can (called be seen simplex in Figure search4, is detailedby default) as follows:is the baseline method in this project. The procedure of the simplex search method,Step as 1: can Methodology be seen in initialization.Figure 4, is detailed as follows: The initial process conditions are preset as X1, which represents the 1st iteration point of the optimization. As each process condition has a different operating range, each is scaled into the same range [0, 100] to facilitate the optimization. The scaled initial conditions are represented as X1. Meanwhile, the coefficients of the methodology, {α, β, γ, δ}, are set at this stage. α is the reflection coefficient, β is the contraction coefficient, γ is the expansion coefficient, δ is the shrink coefficient. Set the coefficients according to the suggestion of Nelder and Mead [29].

Step 2: Initial simplex construction. As shown in Figure5, a particular sequential perturbation strategy is proposed to construct a feasible initial simplex. Set the simplex iteration number s = 0, and set the ﬁrst s vertex V1 = X1. The construction rule for other n vertices is expressed as below: Xk+1 = X1 + τek, X1(k) ≤ 50 s , Vk+1 = Xk+1 , k = 1, ··· , n (3) Xk+1 = X1 − τek, X1(k) > 50

where ek is a column vector, which has 1 in the kth element and zeros in the other ele- ments, Xk+1 represents the (k+1)th iteration point of the optimization; τ is the perturba- Processes 2021, 9, 770 6 of 24

tion coefﬁcient that should be set according to the range of all the process conditions, which has a range of (5, 50). After the perturbation, the initial simplex is constructed as s s s s V = V1 , V2 , ··· , Vn+1 . Experiments are then conducted to obtain the corresponding Processes 2021, 9, x FOR PEER REVIEW s s s s 6 of 26 quality responses vector F = F1, F2, ··· Fn+1 , where F represents the function response for each simplex vertex.

Processes 2021, 9, x FOR PEER REVIEW 7 of 26

FigureFigure 4.4. FlowchartFlowchart ofof thethe simplexsimplex searchsearch method.method.

Xk()Step≤ 50 1: Methodology initialization. 1 τ The initial process conditions are preset as 𝑋, which represents the 1st iteration point of the optimization. As each process condition has a different operating range, each Xk() Xk() is scaled into 1the same range [0,50 100]k+1 to facilitate the optimization. The scaled initial conditions()> are represented as 𝑋. Meanwhile, the coefficients of the methodology, 𝛼, 𝛽, 𝛾, 𝛿, Xk1 50 are set at this stage. 𝛼 is the reflectionτ coefficient, 𝛽 is the contraction coefficient, 𝛾 is the expansion coefficient, ()𝛿 is the shrink coefficient.() Set the coefficients according to the Xkk+1 50 Xk1 suggestion of Nelder and Mead [29]. FigureFigure Step5. 5. InitialInitial 2: Initial simplex simplex simplex construction construction construction. strategy. strategy. As shown in Figure 5, a particular sequential perturbation strategy is proposed to Step 3: Simplex Sorting. constructStep 3: a Simplex feasible Sorting. initial simplex. Set the simplex iteration number s = 0, and set the The simplex iteration number s is updated as s = s + 1. The vertices in the current firstThe vertex simplexs 𝑉 =𝑋 iteration. The constructionnumber s is updatedrule for other as s = n s vertices + 1. The is vertices expressed ins the as below:current simplex V are sorted according to their corresponding quality responses F . The sorted simplex 𝑉 are sorted according=+s∗ τ to their corresponding ≤ quality responses 𝐹 . The simplex may be denotedXXeXkk+11 as V , which,50 has() the following characteristics:  k s* 1 s == sorted simplex may be denoted as V , which has,,1,, theVXkk + 1followingk+1 characteristics:L n (3) =−τs∗ s∗ () >s∗ s∗ XXeXkk+11k≤,501≤ · · · ≤  F1 s**≤≤F2 sssFn ** ≤Fn+1. (4) FF12L FFnn+ 1. (4) e s∗ s∗ whereThus, ∗k V is1 ais column the vertex vector, with which the best has response, 1 in the∗V nkthis element the vertex and with zeros the in next-to-the- the other Thus, 𝑉 is the vertexs∗ with the best response, 𝑉 is the vertex with the next-to-the- worstelements, response, 𝑋 V representsn+1 is the vertex the (k+1) withth the iteration worst response.point of the optimization; 𝜏 is the per- worst response, 𝑉∗ is the vertex with the worst response. turbation coefficient+1 that should be set according to the range of all the process condi- Step 4: Reflection. tions, which has a range of (5, 50). After the perturbation, the initial simplex is con- The reflection operation would be carried out to obtain the reflection point 𝑉 structed as 𝑉 = 𝑉 ,𝑉 ,⋯,𝑉. Experiments are then conducted to obtain the corre- according to the following expression: sponding quality responses vector 𝐹 = 𝐹 ,𝐹 ,⋯𝐹, where F represents the func- s =+ααss − * tion response for each simplex vertex.VVVref (1 ) c n+1 . (5)

∗ 𝑉 is the centroid of all the vertices except 𝑉+1, which is expressed as: n VVnss= ()* / cii=1 . (6)

Experiment to evaluate the quality response 𝐹 at the reflection point. If 𝐹 < 𝐹∗, which means that the performance of the process conditions along the reflection direction may be promising, the expansion operation should be executed; hence, the ∗ ∗ procedure would go to Step 5. If 𝐹 ≤𝐹 ≤𝐹 , which means that the performance of the process conditions along the reflection direction may be neutral, the current reflection point should be accepted, and the worst vertex would be substituted by the ∗ reflection point; thus, the procedure would go the Step 8. If 𝐹 >𝐹 , which means that the performance of the process conditions along the reflection direction may be gloomy, the contraction operation should be conducted, the procedure would go to Step 6. Step 5: Expansion. The expansion operation would be carried out to obtain the expansion point 𝑉 according to the following expression: s =−γγss VVVexp (1 ) cref+ . (7)

Then, experiment to evaluate the quality response 𝐹 at the expansion point. If s s F ≤ F , the expansion is successful, thus 𝑉∗ would be replaced by 𝑉 . Other- exp ref +1 wise, the worst vertex would be substituted by the reflection point. At last, the procedure would go to Step 8. Step 6: Contraction. The contraction operation could be categorized into two types, viz. (i) inside contraction, (ii) outside contraction. Generally, the contraction point 𝑉 could be calculated according to the following formula:

Processes 2021, 9, 770 7 of 24

Step 4: Reflection. s The reflection operation would be carried out to obtain the reflection point Vre f according to the following expression:

s s s∗ Vre f = (1 + α)Vc − αVn+1. (5)

s s∗ Vc is the centroid of all the vertices except Vn+1, which is expressed as: s = n s∗ Vc ∑i=1 Vi /n. (6)

s s s∗ Experiment to evaluate the quality response Fre f at the reflection point. If Fre f < F1 , which means that the performance of the process conditions along the reflection direction may be promising, the expansion operation should be executed; hence, the procedure s∗ s s∗ would go to Step 5. If F1 ≤ Fre f ≤ Fn , which means that the performance of the process conditions along the reflection direction may be neutral, the current reflection point should be accepted, and the worst vertex would be substituted by the reflection point; thus, the s s∗ procedure would go the Step 8. If Fre f > Fn , which means that the performance of the process conditions along the reflection direction may be gloomy, the contraction operation should be conducted, the procedure would go to Step 6. Step 5: Expansion. s The expansion operation would be carried out to obtain the expansion point Vexp according to the following expression:

s s s Vexp = (1 − γ)Vc + γVre f . (7)

s Then, experiment to evaluate the quality response Fexp at the expansion point. If s s s∗ s Fexp ≤ Fre f , the expansion is successful, thus Vn+1 would be replaced by Vexp. Otherwise, the worst vertex would be substituted by the reﬂection point. At last, the procedure would go to Step 8. Step 6: Contraction. The contraction operation could be categorized into two types, viz. (i) inside con- s traction, (ii) outside contraction. Generally, the contraction point Vct could be calculated according to the following formula:

s s s Vct = (1 − β)Vc + βVmax/re f . (8)

s s∗ s where Vmax/re f , the reference point for contraction, maybe Vn+1 or Vre f . The choice of s Vmax/re f depends on the contraction types. s s∗ Case 1: When Fre f ≥ Fn+1, this is inside contraction. s s∗ s s∗ In this case, Vmax/re f = Vn+1, Fmax/re f = Fn+1; conduct experiments to evaluate the s s s∗ quality response Fct. If Fct ≤ Fn+1, the contraction is accepted. s∗ s s∗ Case 2: When Fn ≤ Fre f < Fn+1, this is outside contraction. s s s s In this case, Vmax/re f = Vre f , Fmax/re f = Fre f ; conduct the experiments to evaluate the s s s quality response Fct. If Fct ≤ Fre f , the contraction is accepted. s∗ s If the contraction is accepted in the above cases, replace Vn+1 with Vct and go to Step 8. Otherwise, the contraction is refused, then go to Step 7 for a shrink operation. Step 7: Shrink. The shrink operation could be expressed as:

s∗ s∗ s∗ Vi = (1 − δ)V1 + δVi , i = 2, ··· , n + 1. (9)

All the vertices, except the best vertex, are sequentially updated and substituted by the new vertices generated based on Equation (9). The quality responses of the updated vertices are obtained and updated via experiments. Processes 2021, 9, 770 8 of 24

Step 8: Termination. Since the simplex search is utilized for the quality control, the method would be terminated when the maximum iteration number is achieved or the quality response of the best vertex is satisﬁed within a preset quality tolerance.

3.3. Basic Ideas for Knowledge-Informed Simplex Search Methodology Based on Quasi-Gradient Estimations Considering the genesis and the development process of the simplex search method, it, to some extent, can be regarded as a particular type of knowledge-informed strategy. Different knowledge fusion mechanisms accompany the development of the simplex search method. To achieve more effective integration of process knowledge with the simplex search method, it is necessary to attain some inspiration from its development. This method was initially proposed by Spendley et al. in 1962 [28]. Moreover, its origins can be dated back to Box’s EVOP strategy in 1955 [29]. The basic philosophy of the EVOP method is that the improvement of the industrial process is very similar to the biological evolution process. At the current working point, EVOP continuously sets a series of controlled, slight process parameter variants and observes their responses to find the direction of progressive improvement of the process parameters iteratively. Each advancement of EVOP relies on the knowledge of the current working point and a series of controlled variants at its vicinity, reflecting an intrinsic characteristic of a knowledge- informed mechanism. However, the final optimization operation is still determined by the operation engineers in combination with their experience. To realize the automation of the EVOP, Spendley devised a sequential simplex-design-based optimization methodology, which inherits the basic principle of the EVOP. Under Box’s EVOP methodology, the execution and the search direction of the optimization operation are determined by the operator or the manager. Hence, it is a hybrid optimization strategy relying on both operation knowledge and experience. However, with Spendley’s method, the optimization operation could be determined entirely by the current simplex. The optimization, therefore, gets rid of the constraint of the operator’s experience and becomes automatic and more efficient. However, the iteration of the simplices relies only on the reflection operations in Spendley’s method. The reflection point will substitute the worst vertex, discarded from the current simplex, to form a new simplex. It is a mirror-symmetric reflection that the size of simplices remains unchanged in the optimization process. From the perspective of iterative optimization, the size of the simplex determines the optimal step size. As a result, the optimal step size of the method is kept unchanged. Hence, this is a fixed step size optimization strategy, which seriously restricts its optimization performances. Considering Spendley’s method’s rigidity, Nelder and Mead proposed a revised simplex method that can adapt itself to the optimization’s local landscape. The Nelder-Mead method still follows the idea of sequential simplex design, but it adds additional operations, such as expansion and contraction. These additional operations are conducted depending on the local landscape’s different scenarios; thus, the methodology could adapt to the process landscape. Hence, the revised method becomes a variable-step-size method with the step size of the simplices that could be adjusted dynamically according to the state of the optimization process. This method could contract to the optimal settings more efficiently. The Nelder-Mead method’s significant improvement can be attributed to the deep mining and utilization of knowledge information in the current iteration simplex. This method essentially utilizes the relationships between the reflection points and their subsequent operation points and the present simplex vertices’ knowledge information to realize the step size’s dynamic adjustment. To sum up, the method’s efficiency attribute to the utilization of iteration knowledge and the simplex search method’s development process reveals that making full use of the historical information generated during the optimization process can effectively improve optimization efficiency. Then, from the perspective of using iterative process knowledge to improve the efficiency of MFO, where is the next improvement direction of the simplex search method? Processes 2021, 9, x FOR PEER REVIEW 9 of 26

size of simplices remains unchanged in the optimization process. From the perspective of iterative optimization, the size of the simplex determines the optimal step size. As a result, the optimal step size of the method is kept unchanged. Hence, this is a fixed step size optimization strategy, which seriously restricts its optimization performances. Consider- ing Spendley’s method’s rigidity, Nelder and Mead proposed a revised simplex method that can adapt itself to the optimization’s local landscape. The Nelder-Mead method still follows the idea of sequential simplex design, but it adds additional operations, such as expansion and contraction. These additional operations are conducted depending on the local landscape’s different scenarios; thus, the methodology could adapt to the process landscape. Hence, the revised method becomes a variable-step-size method with the step size of the simplices that could be adjusted dynamically according to the state of the optimization process. This method could contract to the optimal settings more efficiently. The Nelder-Mead method’s significant improvement can be attributed to the deep mining and utilization of knowledge information in the current iteration simplex. This method essentially utilizes the relationships between the reflection points and their subsequent operation points and the present simplex vertices’ knowledge information to realize the step size’s dynamic adjustment. To sum up, the method’s efficiency attribute to the utilization of iteration knowledge and the simplex search method’s development process reveals that making full use of the historical information generated during the optimization process can effectively improve optimization efficiency. Then, from the perspective of using iter- Processes 2021, 9, 770 ative process knowledge to improve the efficiency of MFO, where is the next9 of 24improve- ment direction of the simplex search method? From the idea of knowledge-informed optimization, the traditional simplex search methodFrom is thereviewed. idea of knowledge-informedIt does not make full optimization, use of the historical the traditional iterative simplex information search of the optimizationmethod is reviewed. process. It Some does other not make historical full use knowledge of the historical information iterative generated information during the iterationof the optimization of simplices process. has been Some ignored other before. historical Only knowledge part of the information knowledge generated is utilized. All theduring knowledge the iteration canof be simplices classified has into been two ignored types before. according Only to part their of the relationships knowledge iswith the currentutilized. simplex. All the knowledge The classification can be classified of the intoknowledge two types is according shown in to Figure their relationships 6. The first dimen- sionwith represents the current simplex.the knowledge The classification within the of cu therrent knowledge simplex, is shownwhile inthe Figure second6. The dimension first dimension represents the knowledge within the current simplex, while the second contains the historical knowledge outside the current simplex. dimension contains the historical knowledge outside the current simplex.

FigureFigure 6. ClassiﬁcationClassification of of the the historical historical knowledge knowledg generatede generated by the by simplex the simplex search method.search method.

The traditional simplex search method only utilizes the first dimension knowledge, The traditional simplex search method only utilizes the first dimension knowledge, and excellent optimization performance is achieved with an appropriate knowledge in- andtegration excellent mechanism. optimization Then, performance could the iteration is achieved knowledge with in an the appropriate second dimension knowledge be inte- grationutilized mechanism. to facilitate the Then, optimization could the process? iteration The knowledge answer must in the be second yes. However, dimension the be uti- lizedcritical to question facilitate lies the in optimization how to dig, define, process? and Th finallye answer use this must iteration be yes. knowledge. However, The the critical questionfusion mechanism lies in how of the to iterationdig, define, knowledge, and finally including use this knowledge iteration digging, knowledge. definition, The fusion mechanismand utilization, of the is central iteration to knowledge-informed knowledge, including strategy. knowledge digging, definition, and utilization,The is genesis central and to essential knowledge-informed characteristics of strategy. the simplex search method are analyzed to find sufficientThe genesis knowledge and essential that can characteristics be used to facilitate of the performance simplex improvement.search method The are sim- analyzed plex search method is widely regarded as a gradient-free algorithm. However, as pointed to find sufficient knowledge that can be used to facilitate performance improvement. The out by Spendley, the original EVOP method is essentially a rudimentary steepest descend simplexoptimization search strategy method that is senses widely the steepestregarded descend as a gradient-free direction and climbsalgorithm. down. However, As a as pointedsequential-simplex-design out by Spendley, EVOP the original strategy, EVOP Spendley’s method method is essentially is a kind of a steepest-descend rudimentary steepest method. Its direction is estimated from the simplex’s centroid through the hyperface. The hyperface is opposite to the vertex with the worst response. The advance direction is determined by a rough gradient estimation at the current simplex, as ds = −∇Gs, in essence. For the Nelder-Mead method, the optimization direction is determined by the exact mechanism. The steepest descend direction, which deviates from the actual gradient, is determined solely by the current simplex knowledge. In a sense, this method is still not out of the gradient algorithm’s scope, which could be treated as a special case. The calculation strategy of the operations can be generalized to reveal the essence further. According to the basic principle of the simplex search, the reflection operation could be reviewed and correspondingly transformed into a different form:

s s s∗ s s∗ s Vre f = (1 + α)Vc − αVn+1 = Vc − α Vn+1 − Vc . (10)

The expansion operations could be revised as below: Vs = (1 − γ)Vs + γVs = Vs − γ Vs − Vs = Vs − γαVs∗ − Vs exp c re f c c re f c n+1 c . (11) s s∗ s = Vc − αγ Vn+1 − Vc Processes 2021, 9, 770 10 of 24

The contraction operations could be revised as below:

s s s s s s Vct = (1 − β)Vc + βVmax/re f = Vc − β Vc − Vmax/re f (12)

Hence, the inside contraction and the outside contraction would be transformed into a similar form as below:

s s s s s s∗ s Vinside_ct = Vc − β Vc − Vn+1 = Vc + β Vn+1 − Vc , s s s s s s∗ s V,outside_ct = Vc − β Vc − Vre f = Vc − β α Vn+1 − Vc (13) s s∗ s = Vc − αβ Vn+1 − Vc

From the above equivalent formula, except for shrink operation, other operations of the simplex search method attempt to find a new vertex in a unified way, which can be concluded as below: s s s∗ s Vvertex_new = Vc − ξ Vn+1 − Vc . (14) where ξ can be viewed as the step size of the method, which can be adjusted dynamically according to the current optimization status estimated by the knowledge of the current simplex. According to the general settings, ξ is set to {2, 1, 0.5, −0.5} corresponding to the expansion, reflection, outside contraction, and inside contraction. It is obvious that ξ, in fact, plays the role of the step size. According to the unified iteration strategy, it can be seen that the simplex search method has a mechanism similar to the gradient-based methods. Thus, the vector from the centroid to the worst point is defined as quasi-gradient estimation, which is a rough estimation of the gradient at the centroid of the simplex. The quasi-gradient estimation can be expressed as follows:

s s∗ s Gec = Vn+1 − Vc . (15)

Correspondingly, the iteration strategy of the simplex search method can be represented as below: s s s Vvertex_new = Vc − ξGec. (16) Like the gradient-based method, each simplex during the optimization can be viewed as an iteration point. Hence, the simplex search method with rough gradient estimation behaves like a gradient-based method. Like the SPSA, although the quasi-gradient estimations at every simplex iterations are rough estimations, the method could statistically find the steepest descend route. Thus, its efficiency can be guaranteed. We had proposed a knowledge-informed SPSA based on historical gradient approximations (GK-SPSA) in our previous work. The basic idea of the GK-SPSA lies that if we can appropriately make use of the historical gradient estimation information, then the gradient estimations’ accuracy may be improved to enhance the efficiency of the optimization method. Considering their similarity, the iterative quasi-gradient estimations generated during the simplex search optimization may also be gathered and utilized. This knowledge certainly carries some degree of information on the gradients’ tendency that it may be used to compensate for the gradient estimations at each simplex iteration. Once the gradient estimations’ accuracy could be improved, the simplex search method’s efficiency would be enhanced accordingly. A schematic diagram of the gradient estimation compensation mechanism can be seen in Figure7. Based on the aforementioned knowledge-informed idea, a revised simplex search method, which incorporates the historical quasi gradient estimations to facilitate the optimization, was proposed. The different optimization mechanisms of the traditional simplex search and the revised method are illustrated in Figure8. The revised method utilizes the historical quasi-gradient estimations to compensate for the search direction moderately at each simplex iteration. Like the knowledge-informed SPSA, the revised method was named knowledge-informed simplex search method based on historical quasi gradient estimations (GK-SS for short). This method is a typical knowledge-informed Processes 2021, 9, x FOR PEER REVIEW 11 of 26

Like the gradient-based method, each simplex during the optimization can be viewed as an iteration point. Hence, the simplex search method with rough gradient estimation behaves like a gradient-based method. Like the SPSA, although the quasi- gradient estimations at every simplex iterations are rough estimations, the method could statistically find the steepest descend route. Thus, its efficiency can be guaranteed. We had proposed a knowledge-informed SPSA based on historical gradient approximations (GK-SPSA) in our previous work. The basic idea of the GK-SPSA lies that if we can appropriately make use of the historical gradient estimation information, then the gradient estimations’ accuracy may be improved to enhance the efficiency of the optimization method. Considering their similarity, the iterative quasi-gradient estimations generated during the simplex search optimization may also be gathered and utilized. Processes 2021, 9, 770 This knowledge certainly carries some degree of information on the gradients’ tendency11 of 24 that it may be used to compensate for the gradient estimations at each simplex iteration. Once the gradient estimations’ accuracy could be improved, the simplex search optimizationmethod’s efficiency strategy. would An implementation be enhanced scheme accordingly. for the GK-SSA schematic is proposed diagram and illustratedof the gra- indient detail estimation in the following compensation subsection. mechanism can be seen in Figure 7.

Processes 2021, 9, x FOR PEER REVIEW 12 of 26

FigureFigure 7. A sketch 7. A sketchof the compensation of the compensation mechanism. mechanism.

Based on the aforementioned knowledge-informed idea, a revised simplex search method, which incorporates the historical quasi gradient estimations to facilitate the optimization, was proposed. The different optimization mechanisms of the traditional simplex search and the revised method are illustrated in Figure 8. The revised method utilizes the historical quasi-gradient estimations to compensate for the search direction moderately at each simplex iteration. Like the knowledge-informed SPSA, the revised method was named knowledge-informed simplex search method based on historical quasi gradient estimations (GK-SS for short). This method is a typical knowledge-informed optimization strategy. An implementation scheme for the GK-SS is proposed and illustrated in detail in the following subsection.

(a)

(b)

FigureFigure 8. Optimization 8. Optimization mechanisms mechanisms of the simplex of the search simplex method search illustrated method in a two-dimensional illustrated in a scenario. two-dimensional (a) Traditional simplexscenario. search method. (a) Traditional (b) Revised simplex simplex search search method. method.( b) Revised simplex search method.

3.4. An Implementation Scheme of Knowledge-Informed Simplex Search Method According to the traditional simplex search method principles, GK-SS’s implementation scheme is formulated and detailed based on the above knowledge-informed idea. In this mechanism, the simplex operations should be performed along the compensated search direction, which is different from the traditional method. The simplex search method’s iteration strategy was reformulated from the steepest-descent characteristics of the traditional simplex search just discussed above. An intermediate quantity that has an apparent mathematical meaning was revealed and formulated. This quantity represents a search direction determined by each simplex, which, in a sense, is the approximate gradient information of the current simplex. Thus, this quantity of the current simplex is called the estimated quasi-gradient for

Processes 2021, 9, 770 12 of 24

3.4. An Implementation Scheme of Knowledge-Informed Simplex Search Method According to the traditional simplex search method principles, GK-SS’s implementation scheme is formulated and detailed based on the above knowledge-informed idea. In this mechanism, the simplex operations should be performed along the compensated search direction, which is different from the traditional method. The simplex search method’s iteration strategy was reformulated from the steepest- descent characteristics of the traditional simplex search just discussed above. An intermediate quantity that has an apparent mathematical meaning was revealed and formulated. This quantity represents a search direction determined by each simplex, which, in a sense, is the approximate gradient information of the current simplex. Thus, this quantity of the current simplex is called the estimated quasi-gradient for current simplex (EGCS). The s definition of the EGCS could be expressed as Equation (15), where Gec represents the EGCS at the sth simplex. With the above formula Equations (15) and (16), we construct a kind of quasi-gradient estimation information equivalent to the gradient approximation of the gradient-based strategies for the gradient-free simplex search method. In the traditional simplex search procedure, this kind of information does not exist explicitly. After each simplex is formed and sorted in the GK-SS, the EGCS of the current simplex would first be calculated before the corresponding simplex operations are conducted. Then, it can be recorded into a sequence in an orderly manner. The EGCS sequence, which grows with the simplices’ iteration, stores the historical quasi-gradient estimations generated by each simplex. The EGCS provides a potential link between gradient-free methods and the gradient- based methods. The EGCS, however, is a slightly rough estimation of the gradient at the current simplex due to the operation rule. To integrate the historical knowledge to compensate the gradient estimation accuracy at the current simplex, a new compensated composite gradient for the current simplex (CCG) is defined as follows:

ˆ s ˆ s−1 s ˆ 1 1 Gc = ρsGc + (1 − ρs)Gec, Gc = Gec , (17)

ˆ s ˆ s−1 where Gc is the sth compensated composite gradient (CCG) at the sth simplex, Gc is the ˆ 1 CCG at the (s−1)th simplex, Gc is the CCG at the 1st simplex. ρs is a gradient compensation coefficient at the sth simplex, reflecting the gradient information’s current weights. According to Equation (17), the CCG at the current simplex is determined by all the historical EGCS generated during the previous optimization process. The historical EGCSs of the simplices nearing the current simplex plays a more critical role for the CCG at the current simplex. Considering that the credibility of the EGCS of the neighboring simplex is higher than that far away from the current simplex, the proposed mechanism is reasonable. On the other hand, according to the simplex search method’s characteristics, the accuracy of CCG will gradually improve with the simplex iterations’ development. Hence, the coefficient should not be a fixed constant. Contrarily, it should be a parameter that changes dynamically with the iteration of the simplices. Therefore, a dynamic mechanism for the coefficient can be formed as follows: ∆ρ ρ = ρ − init , (18) s F sτ

where ρF is an upper limit that a CCG can be approached, which is set to 0.5 by default, ∆ρinit is the initial deviation between ρF and its lower limit, which is set to 0.2 by default, and τ is an exponential coefﬁcient, which counts on the iteration number effects on the accuracy of CCG. The coefﬁcient ρs represents the tendency of the cumulative effect of the historical quasi-gradient estimations at each simplex on compensated gradients. From Equation (17), the CCG at the sth simplex is a linear combination of the previous CCG at the (s−1)th simplex and the EGCS at the sth simplex. The historical gradient estimations generated by each simplex are integrated into the revised simplex search Processes 2021, 9, 770 13 of 24

method through the above mechanism. The CCG is utilized to substitute the implicit EGCS, as shown in Equation (16), in the traditional simplex search method’s operations. The

Processes 2021, 9, x FOR PEER REVIEWhistorical quasi-gradient estimations are gained iteratively and stored sequentially.14 of The 26 ratio relationship of the EGCS on the synthesis of the CCG at each iteration is showcased in Figure9.

FigureFigure 9. 9. HistoricalHistorical quasi-gradient quasi-gradient esti estimationmation storage storage and and utilization. utilization.

WithWith the the CCG, CCG, the original reflectionreflection operation,operation, as as shown shown in in Equation Equation (5), (5), would would be besubstituted substituted as below:as below: s s ˆ s Vre f = Vc − αGc, (19) s s ˆ s V ref = V c −αG , (19) where α is the reflection coefficient thatc is kept unchanged. whereThe α expansion is the reflection operation coefficient would that berevised is kept unchanged. accordingly as follows: The expansion operation would be revised accordingly as follows: s s s s s s s V = (1 − γ)V + γV = V − γV + γ V − αGˆ s = V − αγGˆ s. (20) ssssssssexp c re f cs ˆˆc c c c c VVVVVVGVG=−(1γγ ) += − γγ +( c − α) = − αγ. exp crefcc cc c (20) Correspondingly, the outside contraction is: Correspondingly, the outside contraction is: s s s s s s s s ˆ s ˆ s ssssssssVoct = (1 − β)Vc + βVre f = Vc − βVc ˆ+ β Vc − αGc ˆ = Vc − αβGc. (21) VVVVVVGVG=−()1 ββ += − ββα +( c −) = − αβ. oct c ref c c c c c (21) Moreover, the inside contraction is: Moreover, the inside contraction is: ssssssss=−ββs += ( − =) +s + βs −= s =+ +s βˆ − s = s + ˆ s VVVVVVVGict(1 ) V cict n1++11β cVc β()V nn+1 cVc β cVn+1 c . Vc Vc βGc.(22) (22)

TheThe flowchart flowchart of the GK-SSGK-SS isis illustrated illustrated in in Figure Figure 10 10.. As As shown shown in in the the figure, figure, except except for forthe the reflection, reflection, expansion, expansion, and and contraction contraction operations, operations, the the other other operations operations and and the the GK-SS’s GK- SS’sbasic basic procedures procedures are the are same the same as the as traditional the tradit method.ional method. It is natural It is natural since these since operations these op- erationsare the keyare tothe the key simplex to the searchsimplex method. search method. Therefore, Therefore, the revised the simplex revised operationsimplex opera- could tionprovide could an provide appropriate an appropriate mechanism mechanism for the fusion for of the historical fusion of simplex historical information. simplex information. This GK-SS scheme provides a feasible way from a gradient view, which is relatively peculiar for a gradient-free method, for the fusion of the knowledge generated during the optimization process. The primary mechanism of the GK-SS conforms to the GK-SPSA in the search direction compensation. Using knowledge of historical equivalent gradient information generating during the iteration of the simplices, more accurate search directions could be obtained during the optimization. Thus, the optimization efficiency would be enhanced.

Processes 2021, 9, 770 14 of 24 Processes 2021, 9, x FOR PEER REVIEW 15 of 26

FigureFigure 10.10. FlowchartFlowchart ofof thethe knowledge-informedknowledge-informed simplexsimplex searchsearch method.method.

4. ResultsThis GK-SS & Discussions scheme provides a feasible way from a gradient view, which is relatively peculiarA simulation for a gradient-free platform based method, on a for weight the fusion prediction of the model knowledge of a kind generated of post insulator during theis adopted optimization to demonstrate process. Thethe primaryGK-SS’s mechanismperformance of on the the GK-SS quality conforms control of to medium the GK- SPSAvoltage in insulators. the search Based direction on the compensation. platform, a series Using of knowledge deliberately of designed historical tests equivalent are con- gradientducted to information verify the GK-SS’s generating performance during the and iteration characteristics. of the simplices, more accurate search directions could be obtained during the optimization. Thus, the optimization efﬁciency would4.1. Simulation be enhanced. Experimental Setups 4. ResultsAs a typical & Discussions type of medium voltage insulators, a post insulator widely used in electrical engineering was employed to verify the revised knowledge-informed strategy, GK-SS. A simulation platform based on a weight prediction model of a kind of post insulator is The post insulators investigated are made up of epoxy resin, a widely used material for the adopted to demonstrate the GK-SS’s performance on the quality control of medium voltage insulators. In this study, an epoxy molding compound with the brand of Duresco NU 5680V insulators. Based on the platform, a series of deliberately designed tests are conducted to was used. The prototype of the post insulator is shown in Figure 11. Part weight, as a critical verify the GK-SS’s performance and characteristics. quality indicator, was controlled. The target weight of the insulator was set according to the 4.1.product Simulation specification Experimental requirements. Setups Three different types were supposed to be used in the test. The weight target of Type A post insulator was set to 635 g, while Type B 620 g and As a typical type of medium voltage insulators, a post insulator widely used in electricalType C 860g. engineering According was to employedthe process to characteristics verify the revised and control knowledge-informed requirements, a strategy,qualified GK-SS.post insulator’s The post weight insulators bias investigated tolerance is are 0.5 made g. According up of epoxy to the resin, literature a widely review used and material field forinvestigation, the insulators. four Inprocess this study, parameters, an epoxy includ moldinging melt compound temperature, with injection the brand time, of Duresco packing NUtime, 5680 and Vpacking was used. pressure, The prototypehave the most of the sign postificant insulator influence is shownon post ininsulators’ Figure 11 weight.. Part weight,Thus, these as a criticalfour parameters quality indicator, were chosen was controlled.as process parameters The target weightto be optimized, of the insulator repre- T wassented set as according X = [x1, x to2, x the3, x product4] . The descriptions speciﬁcation and requirements. the ranges of Three the parameters different types are werelisted supposedin Table 1.to be used in the test. The weight target of Type A post insulator was set to 635 g, while Type B 620 g and Type C 860 g. According to the process characteristics and

Processes 2021, 9, 770 15 of 24

control requirements, a qualified post insulator’s weight bias tolerance is 0.5 g. According to the literature review and field investigation, four process parameters, including melt temperature, injection time, packing time, and packing pressure, have the most significant influence on post insulators’ weight. Thus, these four parameters were chosen as process Processes 2021, 9, x FOR PEER REVIEW T 16 of 26 parameters to be optimized, represented as X = [x1, x2, x3, x4] . The descriptions and the ranges of the parameters are listed in Table1.

Figure 11. The prototype of the post insulator investigated. Figure 11. The prototype of the post insulator investigated.

Table 1. Process Parameters for Weight Control of a Kind of Post Insulator. Table 1. Process Parameters for Weight Control of a Kind of Post Insulator. Process Parameters Process Variables Range Units Process Parameters Process Variables Range Units Melt Temperature x1 120–255 °C Melt Temperature x 120–255 ◦C Injection Time 1 x2 0.2–0.88 s Injection Time x2 0.2–0.88 s Packing Time x3 3–10 s Packing Time x3 3–10 s 4 PackingPacking Pressure Pressure x4 x 75–9075–90 MPa MPa

With the simplex-search-based quality control framework, as seen in Figure 3, quality With the simplex-search-based quality control framework, as seen in Figure3, quality control of medium voltage insulators relies on online experiments. However, the verifica- control of medium voltage insulators relies on online experiments. However, the verification for the revised method needs a certain amount of sampling experiments, which may tion for the revised method needs a certain amount of sampling experiments, which may be costly if these experiments are to be conducted on the actual process. The surrogate be costly if these experiments are to be conducted on the actual process. The surrogate post insulator platform, which was constructed based on a back-propagation artificial post insulator platform, which was constructed based on a back-propagation artificial neural network (BP-ANN), was employed to advance the efficiency of method verification. neural network (BP-ANN), was employed to advance the efficiency of method verification. Two main reasons are considered, viz. (i) As the key of the paper mainly focused on the Two main reasons are considered, viz. (i) As the key of the paper mainly focused on the effectivenesseffectiveness ofof thethe qualityquality controlcontrol methodology,methodology, thethe utilizationutilization ofof surrogatesurrogate modelsmodels withwith enoughenough accuracyaccuracy wouldwould notnot affectaffect thethe validityvalidity ofof thethe verificationverification forfor thethe qualityquality controlcontrol methods;methods; (ii)(ii) thethe availability availability of of the the simulation simulation platform platform could could shore shore up up a systematic a systematic test test for thefor revisedthe revised method. method. Taking Taking advantage advantage of the of platform, the platform, a systematic a systematic sampling sampling methodology meth- canodology be conducted can be conducted on the simulation on the simulation platform to platform provide to thorough providetest thorough results test at a results low cost. at Sufficienta low cost. experimental Sufficient experimental results can provide results acan relatively provide reliable a relatively performance reliable evaluationperformance in statistics.evaluation The in statistics. detailed information The detailed for informatio the platformn for can the beplatform referred can to [be4]. referred to [4].

4.2.4.2. Results && DiscussionsDiscussions 4.2.1. Effectiveness Test of the GK-SS on Different Application Scenarios 4.2.1.The Effectiveness traditional Test simplex of the search GK-SS method on Different (called Application Traditional Method), Scenarios which is a typical model-freeThe traditional algorithm simplex of the MFO, search is effective method on(called quality Traditional control of batchMethod), processes. which Theis a GK-SS,typical however,model-free as algorithm a revised strategy, of the MFO, still needs is effective to be verifiedon quality for itscontrol effectiveness. of batch pro- The testscesses. at threeThe GK-SS, different however, scenarios as were a revised demonstrated. strategy, still needs to be verified for its effectiveness. The tests at three different scenarios were demonstrated. Quality Control from Different Initial Points QualityFirstly, Control to show from GK-SS’s Different effectiveness Initial Points under any initial conditions, this methodology was testedFirstly, on to quality show GK-SS’s control of effectiveness medium voltage under insulators any initial from conditions, different initialthis method- points. The post insulator of Type A was employed. The part weight specification of the insulator ology was tested on quality control of medium voltage insulators from different initial was 635 ± 0.5 g. Without loss of generality, two initial points were randomly selected for points. The post insulator of Type A was employed. The part weight specification of the demonstration. The first optimization test was conducted from the first initial point the insulator was 635 ± 0.5 g.T Without loss of generality, two initial points were ran- X1 = [181.33, 0.2, 8.74, 77.66] with an initial weight of 642.3 g. The quality target was domly selected for the demonstration. The first optimization test was conducted from the first initial point X1 = [181.33, 0.2, 8.74, 77.66]T with an initial weight of 642.3 g. The quality target was attained with 15 iterations. The optimization trajectory was shown in Figure 12a. It can be seen that the weight of the insulator was adjusted gradually to achieve quality control. Another test from a different initial point, X1 = [202.73, 0.84, 7.82, 81.87]T with an initial weight of 628.9 g, was conducted. The trajectory of quality control was shown in Figure 12b with 22 iterations. As the two tests were started from

Processes 2021, 9, 770 16 of 24

Processes 2021, 9, x FOR PEER REVIEW 17 of 26 attained with 15 iterations. The optimization trajectory was shown in Figure 12a. It can be seen that the weight of the insulator was adjusted gradually to achieve quality control. T Another test from a different initial point, X1 = [202.73, 0.84, 7.82, 81.87] with an initial differentweight ofinitial 628.9 points, g, was conducted.they had different The trajectory optimization of quality trajectories control was and shown different in Figure iter- 12b ationwith numbers. 22 iterations. However, As the twounder tests both were two started cases, from the different target of initial part points, weight they was had achieveddifferent effectively optimization in a trajectorieslimited iteration and different number. iteration numbers. However, under both two cases, the target of part weight was achieved effectively in a limited iteration number.

(a)

(b)

Figure 12. Optimization trajectories of quality control from different initial points: (a) Figure 12. Optimization trajectories of quality control from different initial points: (a) X1 = [181.33, X = [181.33, 0.2, 8.74, 77.66]T,(b)X = [202.73, 0.84, 7.82, 81.87]T. 0.2, 8.74,1 77.66]T, (b) X1 = [202.73, 0.84, 17.82, 81.87]T. To exhibit the optimization trajectory characteristics of the GK-SS further, the iterative To exhibit the optimization trajectory characteristics of the GK-SS further, the it- trajectories of all the four process conditions were demonstrated in Figure 13. The results erative trajectories of all the four process conditions wereT demonstrated in Figure 13. with the initial point, X1 = [202.73, 0.84, 7.82, 81.87] , were showcased. As the process T Theconditions results with have the different initial scales point, in X physical1 = [202.73, quantities, 0.84, 7.82, to observe81.87] , theirwere iteration showcased. tendencies As theat process the same conditions scale, all of have them weredifferent normalized, scales in respectively, physical quantities, to a scaling percentageto observequantity their iterationof 0–100% tendencies according at tothe their same physical scale, bounds.all of them It can were be clearlynormalized, seen that respectively, the ﬁrst ﬁve to itera- a scalingtions werepercentage used for quantity the initial of simplex 0–100% construction according dueto their to that physical the process bounds. parameters It can werebe clearlyperturbed seen onethat by the one first in accordancefive iterations with were the perturbation used for the strategy initial simplex that we proposedconstruction before. due to that the process parameters were perturbed one by one in accordance with the perturbation strategy that we proposed before. From the 6th iteration, the simplices iterated with the knowledge-informed strategy, along with each iteration perturbed

Processes 2021, 9, 770 17 of 24 Processes 2021, 9, x FOR PEER REVIEW 18 of 26

inFrom every the dimension 6th iteration, simultaneously. the simplices As iterated the optimization with the knowledge-informed process progressed, strategy, the param- along eterswith converged each iteration to the perturbed optimal pa inrameter every dimensionsettings iteratively. simultaneously. It was shown As the that optimization the GK- process progressed, the parameters converged to the optimal parameter settings iteratively. SS was effective in quality control of medium voltage insulators from different feasi- It was shown that the GK-SS was effective in quality control of medium voltage insulators ble starting points. from different feasible starting points.

FigureFigure 13. 13. TrajectoriesTrajectories of ofthe the process process parameters parameters of ofknowledge-informed knowledge-informed simplex simplex search search based based on on T T historicalhistorical gradient gradient approximations approximations (GK-SS) (GK-SS) from from X1 X =1 [202.73,= [202.73, 0.84, 0.84, 7.82, 7.82, 81.87] 81.87]. .

QualityQuality Target Target Tracking Tracking Te Testst under under Type Type Transition Transition AsAs a atypical typical batch process,process, thethe post-insulators’ post-insulators’ product product specifications specifications might might frequently fre- quentlychange change due to thedue market to the demandmarket shifting.demand Theshifting. weight The control weight methodology control methodology should have shouldthe flexibility have the toflexibility cope with to cope type with transitions type transitions when the when market the demand market shifts.demand A shifts. weight A setpointweight setpoint tracking tracking test was test conducted was conduc toted further to further demonstrate demonstrate GK-SS’s GK-SS’s effectiveness effective- on nessquality on quality target trackingtarget tracking with type with transition. type transition. Type A Type insulator A insulator was employed was employed with a setpoint with of 635 ± 0.5 g. An initial point X = [229.51, 0.87, 3.92, 83.74]T was selected. The initial a setpoint of 635 ± 0.5g. An initial point1 X1 = [229.51, 0.87, 3.92, 83.74]T was selected. The initialweight weight was 625.9was 625.9 g, which g, which deviated deviated from thefrom target. the target. GK-SS GK-SS was conducted. was conducted. It can It be can seen that the method attained the target weight speedily. GK-SS used 15 iterations to obtain be seen that the method attained the target weight speedily.T GK-SS used 15 iterations to the optimal settings Xopt_Type_A = [182.6, 0.86, 3.25, 85.8] with the part weight of 635.3 g, obtain the optimal settings Xopt_Type_A = [182.6, 0.86, 3.25, 85.8]T with the part weight of 635.3 which meets the current quality specifications. It was supposed that the type B insulator g, which meets the current quality specifications. It was supposed that the type B insulator was required to meet the new market needs at a certain moment. The target weight of type was required to meet the new market needs at a certain moment. The target weight of B was 620 g. At the moment, type transition occurred. To achieve the new target, GK-SS type B was 620 g. At the moment, type transition occurred. To achieve the new target, GK- was implemented. Weight tracking results could be seen in Figure 14. Utilizing the GK-SS, SS was implemented. Weight tracking results could be seen in Figure 14. Utilizing the GK- the weight of the insulator reached the new steady-state within 27 iterations. Finally, the SS, the weight of the insulator reached the new steady-stateT within 27 iterations. Finally, new optimal settings, Xopt_Type_B = [255, 0.88, 4.74, 87.68] with a weight of 620 g, were the new optimal settings, Xopt_Type_B = [255, 0.88, 4.74, 87.68]T with a weight of 620g, were gained. It was indicated that GK-SS was effective in quality setpoint tracking. gained. It was indicated that GK-SS was effective in quality setpoint tracking. Quality Control under Disturbance In the industrial environment, the APG process for the insulators’ production is susceptible to disturbances. When the process is affected by any disturbance, the steady state of the product weight control might be interrupted. The process parameters should be tuned to revert to the target. To test GK-SS’s effectiveness on the robustness of quality control, a disturbance signal was artificially introduced to simulate the disturbance scenario. Type A insulator was used in the test. The target weight of the insulator was 635 g.

Processes 2021, 9, x FOR PEER REVIEW 19 of 26

Processes 2021, 9, x FOR PEER REVIEW 19 of 26 Processes 2021, 9, 770 18 of 24

Figure 14. Weight target tracking trajectory from a randomly selected initial point X1 = [229.51, 0.87, 3.92, 83.74]T.

Quality Control under Disturbance In the industrial environment, the APG process for the insulators’ production is susceptible to disturbances. When the process is affected by any disturbance, the steady state of the product weight control might be interrupted. The process parameters should be tuned to revert to the target. To test GK-SS’s effectiveness on the robustness of quality control, a disturbance signal was artificially introduced to simulate the disturbance sce- Figure 14. Weight target tracking trajectory from a randomly selected initial point nario.Figure 14.Type Weight A insulator target tracking was trajectoryused in frtheom test. a randomly The target selected weight initial pointof the X 1insulator = [229.51, was 635 g. X = [229.51, 0.87, 3.92, 83.74]T. 0.87,1 3.92,When 83.74] theT. quality of epoxy resin fluctuates, the weight of the post under the same processWhen conditions the quality will of drift epoxy from resin the ﬂuctuates, target. Without the weight loss of of the generality, post under the the part same weight Quality Control under Disturbance modelprocess was conditions added a will weight drift fromdeviation the target. by 20 Without g intentionally loss of generality, to simulate the partthe weight offset scene.modelIn Subsequently,the was industrial added a environment, weight the weight deviation theof APGthe by 20insulator process g intentionally for under the insulators’ tothe simulate previous production the process weight is offsetsus- conditions deviatedceptiblescene. to Subsequently, from disturbances. the target. the When weightThe the weight process of the deviation insulator is affected undercould by any the be disturbance, previousseen in Figure process the steady 15. conditions When state the dis- turbanceofdeviated the product occurred, from weight the target.GK-SS controlThe was might weight reactivated. be deviationinterrupted. It could The bebeprocess seenseen in parametersfrom Figure Figure 15 .should When 15 that be the GK-SS returnedtuneddisturbance to revertthe occurred,weight to the totarget. GK-SS the targetTo was test reactivated. within GK-SS’s only effectiveness It couldten iterations. be seen on the from It robustness was Figure indicated 15 ofthat quality GK-SS that the ro- control, a disturbance signal was artificially introduced to simulate the disturbance sce- bustnessreturned of the GK-SS weight to tocope the with target weight within disturbance only ten iterations. was effective. It was indicated that the nario.robustness Type A of insulator GK-SS to was cope used with in weight the test. disturbance The target wasweight effective. of the insulator was 635 g. When the quality of epoxy resin fluctuates, the weight of the post under the same process conditions will drift from the target. Without loss of generality, the part weight model was added a weight deviation by 20 g intentionally to simulate the weight offset scene. Subsequently, the weight of the insulator under the previous process conditions deviated from the target. The weight deviation could be seen in Figure 15. When the disturbance occurred, GK-SS was reactivated. It could be seen from Figure 15 that GK-SS returned the weight to the target within only ten iterations. It was indicated that the robustness of GK-SS to cope with weight disturbance was effective.

FigureFigure 15. 15. QualityQuality control underunder disturbance. disturbance.

4.2.2. Efﬁciency Test of the GK-SS

The GK-SS was showcased to be effective on the quality control of medium voltage insulators from the above tests. As this knowledge-informed methodology was proposed to improve quality control efﬁciency, the critical question is whether the revised method could achieve the expected goal: to reduce the cost of quality control.

Figure 15. Quality control under disturbance. GK-SS vs. GK-SPSA To verify the efﬁciency of the GK-SS relative to the GK-SPSA, the Latin Hypercube Sampling (LHS), which was a popular statistical method for generating a near-random

Processes 2021, 9, x FOR PEER REVIEW 20 of 26

4.2.2. Efficiency Test of the GK-SS The GK-SS was showcased to be effective on the quality control of medium voltage insulators from the above tests. As this knowledge-informed methodology was proposed to improve quality control efficiency, the critical question is whether the revised method could achieve the expected goal: to reduce the cost of quality control.

GK-SS vs. GK-SPSA To verify the efficiency of the GK-SS relative to the GK-SPSA, the Latin Hyper- cube Sampling (LHS), which was a popular statistical method for generating a near- Processes 2021, 9, 770 random sample of process conditions from a multidimensional distribution,19 ofwas 24 utilized. A sample of 100 different initial points was generated. The GK-SS and GK-SPSA were conducted on the sample, respectively, and the experiment results were rec- ordedsample and of analyzed process conditions accordingly. from a multidimensionalIn the test, Type distribution, C with a waspart utilized. weight A target sample of 860 of 100 different initial points was generated. The GK-SS and GK-SPSA were conducted g was employed. A randomly selected optimization result was demonstrated in Fig- on the sample, respectively, and the experiment results were recorded and analyzed ureaccordingly. 16, which Inillustrated the test, Typethe typical C with acharacteristics part weight target of the of 860GK-SS g was and employed. GK-SPSA. A The resultrandomly showed selected that optimizationGK-SS achieved result wasthe goal demonstrated of quality in Figureoptimization 16, which more illustrated quickly the than GK-SPSA.typical characteristics The statistical of the results GK-SS andcould GK-SPSA. verify Thethe resultefficiency showed of that the GK-SS GK-SS achieved relative to GK-SPSA.the goal of Figure quality 17a optimization illustrates more the quickly average than iteration GK-SPSA. number The statistical of the results GK-SPSA could and GK-SS.verify It the can efﬁciency be observed of the GK-SSthat the relative average to GK-SPSA. iteration Figure number 17a illustratesdiminished the significantly average iteration number of the GK-SPSA and GK-SS. It can be observed that the average iteration bynumber the GK-SS diminished with a signiﬁcantlyreduction of by 43.91%. the GK-SS Figure with a17b reduction demonstrates of 43.91%. the Figure performance 17b statisticsdemonstrates on the the superior, performance inferior, statistics and onequa thel ratios superior, that inferior, the GK-SS and equal relative ratios to that the GK- SPSA.the GK-SS It can relative be seen to thethat GK-SPSA. the GK-SS It can behaved be seen thatbetter the in GK-SS 69% behaved of cases, better while in 69%the GK- SPSAof cases, behaved while better the GK-SPSA only in behaved 25% of better cases. only The in 25%statistical of cases. results The statistical indicated results that the GK-SSindicated acted that better the GK-SSthan the acted GK-SPSA better than on the medium GK-SPSA voltage on medium insulators’ voltage quality insulators’ control. quality control.

Processes 2021, 9, x FOR PEER REVIEW 21 of 26 Figure 16. Quality control trajectories via GK-SS and knowledge-informed simultaneous perturbation Figure 16. Quality control trajectories via GK-SS and knowledge-informed simultaneous stochastic approximation based on historical gradient approximations (GK-SPSA). perturbation stochastic approximation based on historical gradient approximations (GK- SPSA).

(a) (b)

Figure 17. Performance evaluation of GK-SS vs. GK-SPSA by statistics. (a) Averaged iteration number. (b) Superiority ratio Figure 17. Performance evaluation of GK-SS vs. GK-SPSA by statistics. (a) Averaged iteration number. (b) Superiority ratioby directby direct performance performance comparison. comparison.

GK-SS vs. Traditional Simplex Search Method The efficiency improvement of the GK-SS relative to the traditional simplex search method should be verified further. The quality control methods’ relative performance efficiency should be a statistical efficiency under the initial points of the whole feasible region. To compare the GK-SS’s performance efficiency and the traditional method, appropriate experiments should be deliberately designed to provide a group of sufficient, randomly distributed, and representative initial points for quality control simulation tests. The performance should be evaluated by estimating their comprehensive performance indicators under these tests. Under all test samples, the average number of iterations of the two methods and the improved method’s performance improvement ratio can be utilized as the performance evaluation indices for the GK-SS method. In this framework, the quality control process was treated as a system, as illustrated in Figure 18. The system’s input is the optimization method to be tested, and the output is the response of the performance efficiency index. The representative initial points generated by the design of experiments, which are the controllable variables in this test framework, represent the process conditions’ feasible region. The LHS was adopted for generating a sampling of initial points. In LHS, a Latin square is a square grid in which there is only one sample in each row and column. For a problem with N variables, with each variable be divided into M equally intervals, there will be MN hypercubes. However, with the LHS, only M independent samples are generated. With the help of LHS, the initial points in experimental design could satisfy the requirements of the characteristics of randomly distributed and representative. As the number of sampling points in a single LHS is relatively limited, the initial points’ coverage cannot be comprehensively guaranteed. To ensure the comprehensive coverage of the feasible region on the basis of the LHS method, a sequential LHS scheme was proposed. A series of LHS batches constitute this strategy. A single LHS batch is utilized to generate a basic unit of independent initial points with a fixed number. The performance index could be evaluated at each LHS batch. A series of LHS batches are conducted iteratively in a sequential way. With this strategy, the tendency of the performance index could be observed. Subsequently, the gradual change of performance evaluation index with the increase of LHS batches could be evaluated and analyzed.

Processes 2021, 9, 770 20 of 24

GK-SS vs. Traditional Simplex Search Method The efficiency improvement of the GK-SS relative to the traditional simplex search method should be verified further. The quality control methods’ relative performance efficiency should be a statistical efficiency under the initial points of the whole feasible region. To compare the GK-SS’s performance efficiency and the traditional method, appropriate experiments should be deliberately designed to provide a group of sufficient, randomly distributed, and representative initial points for quality control simulation tests. The performance should be evaluated by estimating their comprehensive performance indicators under these tests. Under all test samples, the average number of iterations of the two methods and the improved method’s performance improvement ratio can be utilized as the performance evaluation indices for the GK-SS method. In this framework, the quality control process was treated as a system, as illustrated in Figure 18. The system’s input is the optimization method to be tested, and the output is the response of the performance efficiency index. The representative initial points generated by the design of experiments, which are the controllable variables in this test framework, represent the process conditions’ feasible region. The LHS was adopted for generating a sampling of initial points. In LHS, a Latin square is a square grid in which there is only one sample in each row and column. For a problem with N variables, with each variable be divided into M equally intervals, there will be MN hypercubes. However, with the LHS, only M independent samples are generated. With the help of LHS, the initial points in experimental design could satisfy the requirements of the characteristics of randomly distributed and representative. As the number of sampling points in a single LHS is relatively limited, the initial points’ coverage cannot be comprehensively guaranteed. To ensure the comprehensive coverage of the feasible region on the basis of the LHS method, a sequential LHS scheme was proposed. A series of LHS batches constitute this strategy. A single LHS batch is utilized to generate a basic unit of independent initial points with a fixed Processes 2021, 9, x FOR PEER REVIEWnumber. The performance index could be evaluated at each LHS batch. A series of LHS 22 of 26 batches are conducted iteratively in a sequential way. With this strategy, the tendency of the performance index could be observed. Subsequently, the gradual change of performance evaluation index with the increase of LHS batches could be evaluated and analyzed. The Theconclusion conclusion on the on performance the performance efficiency couldefficiency be given could based be on given the changing based on trend the and changing trendthe stability and the of thestability performance of the indices. performance indices.

FigureFigure 18. The generalgeneral framework framework for for performance performance efﬁciency efficiency evaluation evaluation of different of different methods. methods.

InIn this test, test, the the dimension dimension of the of processthe process conditions conditions of the initial of the points initial is 4. points Consider is 4. Con- sideradopting adopting a reasonable a reasonable level for the level process for conditions,the process each conditions, process condition each process was equally condition divided into ten levels. For a single LHS, 10 independent sampling points would be wasgenerated equally randomly. divided To into evaluate ten levels. the performance For a single of anLHS, optimization 10 independent run objectively, sampling the points woulditeration be number generated of the randomly. transient process To evaluate on quality the control performance was selected of as an a critical optimization index run objectively,to show the method’sthe iteration performance number on of a the single transient run. The process average on iteration quality number control will was se- lectedbe calculated as a critical for an index LHS batchto show to reﬂect the meth the twood’s methods’ performance performance on a single on this run. batch. The aver- ageFurthermore, iteration thenumber superior, will inferior, be calculated and the equal for numberan LHS of batch the quality to reflect control the runs, two that methods’ the GK-SS relative to the traditional method on the performance, is recorded. performance on this batch. Furthermore, the superior, inferior, and the equal number of the quality control runs, that the GK-SS relative to the traditional method on the performance, is recorded. A certain number of LHS batches will be sequentially conducted to ensure enough coverage on the initial points. In this test, the batch number of the sequential design was set to 100. That is a 100 LHS batch with 1000 runs, which provides a group of sufficient, randomly distributed, and representative initial points. All the experiments were carried out under identical conditions, with only the difference in the initial points. The results of every LHS batches were observed. The dot diagram of the averaged iteration number for each LHS batch was shown in Fig- ure 19a. It can be seen from the figure that the average iteration number per LHS batch varied considerably. Even though the GK-SS behaved inferior in some batches, it costed relatively fewer iterations in most cases. To evaluate their relative costs on iterations clearly, the box plot of the averaged iteration number at the two methods was illustrated in Figure 19b. It can be seen that the averaged iteration number was significantly decreased with the revised GK-SS.

(a) (b)

Processes 2021, 9, x FOR PEER REVIEW 22 of 26

The conclusion on the performance efficiency could be given based on the changing trend and the stability of the performance indices.

Figure 18. The general framework for performance efficiency evaluation of different methods.

In this test, the dimension of the process conditions of the initial points is 4. Con- sider adopting a reasonable level for the process conditions, each process condition was equally divided into ten levels. For a single LHS, 10 independent sampling points would be generated randomly. To evaluate the performance of an optimization run objectively, the iteration number of the transient process on quality control was selected as a critical index to show the method’s performance on a single run. The average iteration number will be calculated for an LHS batch to reflect the two methods’ performance on this batch. Furthermore, the superior, inferior, and the equal number of the quality control runs, that the GK-SS relative to the traditional method on the performance, is recorded. Processes 2021, 9, 770 A certain number of LHS batches will be sequentially conducted 21to of 24ensure enough coverage on the initial points. In this test, the batch number of the sequential design was set to 100. That is a 100 LHS batch with 1000 runs, which provides a group of sufficient,A certain randomly number ofdistributed, LHS batches and will representative be sequentiallyconducted initial points. to ensure enough coverage on the initial points. In this test, the batch number of the sequential design was All the experiments were carried out under identical conditions, with only the set to 100. That is a 100 LHS batch with 1000 runs, which provides a group of sufficient, differencerandomly in distributed, the initial and points. representative The result initials of points. every LHS batches were observed. The dot diagramAll the of experiments the averaged were iteration carried out number under identical for each conditions, LHS batch with was only shown the dif- in Fig- ureference 19a. It in can the be initial seen points. from the The figure results th ofat every the average LHS batches iteration were number observed. per The LHS dot batch varieddiagram considerably. of the averaged Even iteration though number the GK- for eachSS behaved LHS batch inferior was shown in insome Figure batches, 19a. it costedIt can relatively be seen from fewer the iterations figure that in the most average cases. iteration To evaluate number pertheir LHS relative batch variedcosts on it- considerably. Even though the GK-SS behaved inferior in some batches, it costed relatively erationsfewer iterationsclearly, inthe most box cases. plot To of evaluate the averaged their relative iteration costs onnumber iterations at clearly,the two the methods box wasplot illustrated of the averaged in Figure iteration 19b. number It can atbe the seen two that methods the averaged was illustrated iteration in Figure number 19b. was significantlyIt can be seen decreased that the averaged with the iteration revised number GK-SS. was significantly decreased with the revised GK-SS.

(a) (b)

Figure 19. Average iteration number distribution in the test: (a) dot diagram of the average iteration numbers and (b) box plot of the averaged iteration number.

The sequential LHS design provided an angle of view to observe the change of the performance indices and the increase of independent initial points. Figure 20a showed the trajectories of the accumulated average iteration number of the GK-SS and the traditional method. Unlike the above Figure 19a, this figure showed a clear tendency of the accumulated averaged iteration number, which was developed with more initial points of the feasible region covered. It can be seen clearly that the GK-SS costed relatively fewer iteration numbers than the traditional method. Figure 20b showed the trajectory of GK-SS’s accumulated reduction ratio on the iteration number. The reduction ratio of the GK-SS on the iteration number finally converged to about 10.56%. The results have shown that the GK-SS could reduce the traditional method’s optimization cost by about 10% in a statistical sense. The cumulative effect eliminates the randomness of single LHS batches and presents a more smooth performance accurately. It can be seen that in the initial LHS batches, due to the limited coverage of the total samples, there are some fluctuations in the trajectories of the above two cumulative indexes. With the proceeding of LHS batches, the number of samples covered increased gradually. Finally, the cumulative performance indices converged to a stable value. Under the cumulative effect, the average performance of quality control in the whole feasible region exhibited convergence characteristics, and the variability was thus eliminated. Through the sequential LHS scheme, the changing trend of the performance evaluation indexes of the two methods could be monitored dynamically. The statistical results can provide the basis for the performance evaluation of the two methods. Processes 2021, 9, x FOR PEER REVIEW 23 of 26

Figure 19. Average iteration number distribution in the test: (a) dot diagram of the average iteration numbers and (b) box plot of the averaged iteration number.

The sequential LHS design provided an angle of view to observe the change of the performance indices and the increase of independent initial points. Figure 20a showed the trajectories of the accumulated average iteration number of the GK-SS and the traditional method. Unlike the above Figure 19a, this figure showed a clear tendency of the accumulated averaged iteration number, which was developed with more initial points of the feasible region covered. It can be seen clearly that the GK-SS costed relatively fewer iteration numbers than the traditional method. Figure 20b showed the trajectory of GK-SS’s accumulated reduction ratio on the iteration number. The reduction ratio of the GK-SS on the iteration number finally converged to about 10.56%. The results have shown that the GK-SS could reduce the traditional method’s optimization cost by about 10% in a statistical sense. The cumulative effect eliminates the randomness of single LHS batches and presents a more smooth performance accurately. It can be seen that in the initial LHS batches, due to the limited coverage of the total samples, there are some fluctuations in the trajectories of the above two cumulative indexes. With the proceeding of LHS batches, the number of samples covered increased gradually. Finally, the cumulative performance indices converged to a stable value. Under the cumulative effect, the average performance of quality control in the whole feasible region exhibited convergence characteristics, and the variability was thus eliminated. Through the sequential LHS scheme, the changing trend of the performance evaluation indexes of the two methods could be monitored dynamically. The statistical results can provide the basis for the performance evaluation of the two Processes 2021, 9, 770 22 of 24 methods.

(a) (b)

Figure 20. The tendency with the sequential Latin Hypercube Sampling LHS. (a) Accumulated average iteration number Figure 20. The tendency with the sequential Latin Hypercube Sampling LHS. (a) Accumulated average iteration number trajectory; (b) Reduction ratio trajectory of GK-SS on the iteration number. trajectory; (b) Reduction ratio trajectory of GK-SS on the iteration number. To sum up, the performance efficiency indices were demonstrated in Figure 21. Down- wardTo sum trends up, in iterationthe performance number statistics efficiency of the indices GK-SS relative were demonstrated to the traditional in simplex Figure 21. Downwardsearch method trends suggested in iteration that thenumber revised statistics method isof relatively the GK-SS efficient. relative Figure to the 21 atraditional illus- simplextrates thesearch average method iteration suggested number that with the the tworevised methods. method It can is relatively be observed efficient. that the Fig- average iteration number diminished significantly, with a reduction of 10.56%. Figure 21b ure 21a illustrates the average iteration number with the two methods. It can be ob- demonstrates the performance statistics on the superior, inferior, and equal ratios that servedthe GK-SS that the relative average to the iteration traditional number method. diminished It can be seen significantly, that the GK-SS with behaved a reduction not of 10.56%.inferior toFigure the traditional 21b demonstrates method in the almost performance 71% of cases. statistics Drawing on athe contrast, superior, GK-SS inferior, andacted equal better ratios in ofthat 48% the cases, GK-SS and relative the traditional to the method traditional behaved method. better It only can in be 28% seen of that thecases. GK-SS Therefore, behaved the not proportion inferior of to the the GK-SS trad behaveditional superiormethod is in 1.71 almost times that71% of of the cases. Drawingtraditional a contrast, method. Accordingly,GK-SS acted the better results in showed of 48% that cases, the knowledge-informed and the traditional mecha- method nism had an appreciable effect on the quality control methods, and the conclusion can be behaveddrawn thatbetter the only GK-SS in behaved 28% of bettercases. than Ther theefore, traditional the proportion method. Hence, of the it was GK-SS indicated behaved that the knowledge-informed mechanism is effective for the quality control of medium voltage insulators.

(a) (b)

Figure 21. Performance evaluation of GK-SS vs. traditional simplex search method by statistics. (a) Averaged iteration 1 number. (b) Superiority ratio by direct performance comparison. Processes 2021, 9, 770 23 of 24

5. Conclusions In this study, the idea of a knowledge-informed optimization strategy was extended to another kind of gradient-free algorithm—the simplex search method. The knowledge that the simplex search method generated during the optimization process was investigated. A retrospect on the development course of the simplex search from a view of knowledge utilization, which provides some general acquaintance with the method’s essence, was proposed. Furthermore, the simplex search method was reviewed as a type of steepest descent methodology that a new mathematical quantity-quasi-gradient estimation was constructed. Based on the simplex search principle and the knowledge-informed idea, a revised strategy based on the utilization of the historical quasi-gradient estimations, GK- SS, was proposed and formulated with a feasible knowledge fusion mechanism. Typical illustrative tests were conducted on a post insulator’s weight control to confirm that the GK-SS was effective and efficient. The experimental results have shown that the GK-SS could significantly reduce the effort required to attain quality control. The knowledge- informed simplex search can be investigated deeply on its theoretical analysis and the construction of more efficient knowledge fusion mechanisms. As a general optimization methodology, the idea and the general framework of the GK-SS can be extended to other similar industrial applications.

Author Contributions: Conceptualization, X.K.; Data curation, X.K. and D.Z.; Formal analysis, X.K.; Funding acquisition, X.K.; Investigation, X.K. and D.Z.; Methodology, X.K.; Project administration, X.K.; Resources, X.K.; Software, X.K.; Supervision, X.K.; Validation, X.K. and D.Z.; Visualization, X.K.; Writing—original draft, X.K.; Writing—review & editing, X.K. and D.Z. All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by the Natural Science Foundation of Fujian Province, grant number 2018J01564 and 2019J01867, and Science and Technology Program of Xiamen, grant number 3502Z20179036, and the Science and Technology Program of Longyan, grant number 2018LYF7006. Data Availability Statement: No new data were created or analyzed in this study. Data sharing is not applicable to this article. Conﬂicts of Interest: The authors declare no conﬂict of interest.

References 1. Qiao, X.; Zhang, Z.; Jiang, X.; He, Y.; Li, X. Application of Grey Theory in Pollution Prediction on Insulator Surface in Power Systems. Eng. Fail. Anal. 2019, 106, 104153. [CrossRef] 2. Tian, W.C.; Yu, W.B.; Shi, J.; Wang, Y.K. The Property, Preparation and Application of Topological Insulators: A Review. Materials 2017, 10, 45. [CrossRef] 3. Konig, D.; Reichert, R.; Vlase, I.O.; Hubler, E.; Massen, U. Recent Progress in Computer Simulation Applied in the Automatic Pressure Gelation (APG) Process for Epoxy Casting Systems. In Proceedings of the Electrical/Electronics Insulation Conference, Chicago, IL, USA, 4–7 October 1993; pp. 17–23. 4. Kong, X.S.; Guo, J.M.; Zheng, D.B.; Zhang, J.; Fu, W. Quality Control for Medium Voltage Insulator via a Knowledge-Informed SPSA Based on Historical Gradient Approximations. Processes 2020, 8, 146. [CrossRef] 5. Kong, X.S.; Guo, J.M.; Zheng, D.B.; Yang, J.F.; Zhang, J.; Fu, W. An Improved-SPSA Quality Control Method for Medium Voltage Insulator SPSA. J. Chem. Eng. Chin. Univ. 2020, 34, 1500–1510. 6. Gehrig, M. “Rapid APG”, A New Technique to Reduce Cycle Times in the Processing of Epoxy Casting Systems. In Proceedings of the Electrical Insulation Conference and Electrical Manufacturing and Coil Winding Conference, Rosemont, IL, USA, 25–25 September 1997; pp. 23–29. 7. Kong, X.S.; Yang, Y.; Chen, X.; Shao, Z.J.; Gao, F.R. Quality Control via Model-Free Optimization for a Type of Batch Process with a Short Cycle Time and Low Operational Cost. Ind. Eng. Chem. Res. 2011, 50, 2994–3003. [CrossRef] 8. Zhu, S.; Yang, Y.; Yang, B.; Shao, Z.; Chen, X. Model-Free Quality Optimization Strategy for a Batch Process with Short Cycle Time and Low Operational Cost. Ind. Eng. Chem. Res. 2014, 53, 16384–16396. [CrossRef] 9. Yang, Y.; Yang, B.; Zhu, S.; Chen, X. Online Quality Optimization of the Injection Molding Process via Digital Image Processing and Model-Free Optimization. J. Mater. Process. Technol. 2015, 226, 85–98. [CrossRef] 10. Zhao, J.; Yang, Y.; Chen, X.; Gao, F. An Iterative Modeling and Trust-Region Optimization Method for Batch Processes. Ind. Eng. Chem. Res. 2015, 54, 3186–3199. [CrossRef] 11. Zhao, F.; Lu, N.; Lu, J. Quality Control of Batch Processes Using Natural Gradient Based Model-Free Optimization. Ind. Eng. Chem. Res. 2014, 53, 17419–17428. [CrossRef] Processes 2021, 9, 770 24 of 24

12. Spall, J.C. Feedback and Weighting Mechanisms for Improving Jacobian (Hessian) Estimates in the Adaptive Simultaneous Perturbation Algorithm. IEEE Trans. Autom. Con. 2009, 54, 1216–1229. [CrossRef] 13. Dong, N.; Han, X.; Gao, Z.; Chen, Z.; Wu, A. SPSA-Based Data-Driven Control Strategy for Load Frequency Control of Power Systems. IET Gener. Transm. Distrib. 2018, 12, 414–422. [CrossRef] 14. Hou, Z.S.; Wang, Z. From Model-Based Control to Data-Driven Control: Survey, Classification and Perspective. Inf. Sci. 2013, 235, 3–35. [CrossRef] 15. Lei, T.; Hou, Z.; Ren, Y. Data-Driven Model Free Adaptive Perimeter Control for Multi-Region Urban Traffic Networks With Route Choice. IEEE Trans. Intell. Transp. Syst. 2020, 21, 2894–2905. [CrossRef] 16. Cao, R.; Hou, Z.; Zhao, Y.; Zhang, B. Model Free Adaptive Iterative Learning Control for Tool Feed System in Noncircular Turning. IEEE Access 2019, 7, 113712–113725. [CrossRef] 17. Xing, L.N.; Rohlfshagen, P.; Chen, Y.W.; Yao, X. A Hybrid Ant Colony Optimization Algorithm for the Extended Capacitated Arc Routing Problem. IEEE Trans. Syst. Man. Cybern. 2011, 41, 1110–1123. [CrossRef] 18. Song, Y.J.; Ma, X.; Li, X.J.; Xing, L.N.; Wang, P. Learning-Guided Nondominated Sorting Genetic Algorithm II for Multi-Objective Satellite Range Scheduling Problem. Swarm. Evol. Comput. 2019, 49, 194–205. [CrossRef] 19. Kong, X.; Guo, J.; Zheng, D.; Zhang, J.; Jiang, S.; Fu, W. An Improved SPSA-Based Quality Control Method for Medium Voltage Insulation Parts. In Proceedings of the Chinese Process System Engineering Conference, Hangzhou, China, 25–28 October 2019. 20. Chang, K.H. Stochastic Nelder–Mead Simplex Method–A New Globally Convergent Direct Search Method for Simulation Optimization. Eur. J. Oper. Res. 2012, 220, 684–694. [CrossRef] 21. Chen, W.; Cao, Y.; Cheng, S.; Sun, Y.; Liu, Q.; Li, Y. Simplex Search-Based Brain Storm Optimization. IEEE Access 2018, 6, 75997–76006. [CrossRef] 22. Abdelhalim, A.; Nakata, K.; El-Alem, M.; Eltawil, A. A Hybrid Evolutionary-Simplex Search Method to Solve Nonlinear Constrained Optimization Problems. Soft Comput. 2019, 23, 12001–12015. [CrossRef] 23. Barton, R.R.; Ivey, J.S.J.M.S., Jr. Nelder-Mead Simplex Modifications for Simulation Optimization. Manag. Sci. 1996, 42, 954–973. [CrossRef] 24. Tomick, J.J.; Arnold, S.F.; Barton, R.R. Sample Size Selection for Improved Nelder-Mead Performance. In Proceedings of the Winter Simulation Conference Proceedings, Arlington, VA, USA, 3–6 December 1995; pp. 341–345. 25. Kelley, C.T. Detection and Remediation of Stagnation in the Nelder–Mead Algorithm Using a Sufficient Decrease Condition. SIAM J. Optim. 1999, 10, 43–55. [CrossRef] 26. Spall, J.C.; Nowak, W. Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control. IEEE Trans. Neu. Net. 2007, 18, 964–965. 27. Box, G.E.P. Evolutionary Operation: A Method for Increasing Industrial Productivity. J. R. Stat. Society: Ser. C (Appl. Stat.) 1957, 6, 81–101. [CrossRef] 28. Spendley, W.; Hext, G.R.; Himsworth, F.R. Sequential Application of Simplex Designs in Optimization and Evolutionary Operations. Technometrics 1962, 4, 441–461. [CrossRef] 29. Nelder, J.A.; Mead, R. A Simplex-Method for Function Minimization. Comput. J. 1965, 7, 308–313. [CrossRef]