Available online at www.sciencedirect.com ScienceDirect
Procedia Computer Science 103 ( 2017 ) 14 – 19
XIIth International Symposium Intelligent Systems, INTELS16, 5-7 October 2016, Moscow, Russia Grammatical evolution for neural network optimization in the control system synthesis problem D.E. Kazaryan∗, A.V. Savinkov
RUDN University, Miklukho-Maklaya str. 6, Moscow 117198, Russia
Abstract
Grammatical evolution is a perspective branch of the genetic programming. It uses evolutionary algorithm based search engine and Backus — Naur form of domain-specific language grammar specifications to find symbolic expressions. This paper describes an application of this method to the control function synthesis problem. Feed-forward neural network was used as an approximation of the control function, that depends on the object state variables. Two-stage algorithm is presented: grammatical evolution optimizes neural network structure and genetic algorithm tunes weights. Computational experiments were performed on the simple kinematic model of a two-wheel driving mobile robot. Training was performed on a set of initial conditions. Results show that the proposed algorithm is able to successfully synthesize a control function. ©c 20172017 TheThe Authors. Authors. Published Published by byElsevier Elsevier B.V. B.V. This is an open access article under the CC BY-NC-ND license Peer-revie(http://creativecommons.org/licenses/by-nc-nd/4.0/w under responsibility of the scientific). committee of the XIIth International Symposium “Intelligent Systems”. Peer-review under responsibility of the scientific committee of the XIIth International Symposium “Intelligent Systems” Keywords: grammatical evolution; control system synthesis; artificial neural networks.
1. Introduction
Control synthesis is a complex problem, that usually involves a great amount of analytical computations, especially for the nonlinear problems, or comprised of tedious uniform tasks. For nonlinear problems common approach is to linearize a plant around operating points and then design a linear controller. This approach could lead to oversimpli- fication of the model. These issues will be avoided if computer take the brunt of a solution search. Artificial neural networks (ANNs) are often used in control applications as they are developed theoretically as well as have computationally effective implementations. There were successful attempts to use ANN as a nonlinear controller, that takes the plant state variables as inputs and produce control signal 1,2. Arguably the most widely used model of ANNs in control applications is nonlinear autoregressive model with exogenous inputs (NARX), that uses delayed inputs and outputs: h(k) = F y(k − 1), y(k − 2),...,y(k − n), u(k), u(k − 1),...,u(k − m) . (1)
∗ Corresponding author. Tel.: +7-495-955-0792. E-mail address: kazaryan [email protected]
1877-0509 © 2017 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the scientific committee of the XIIth International Symposium “Intelligent Systems” doi: 10.1016/j.procs.2017.01.002 D.E. Kazaryan and A.V. Savinkov / Procedia Computer Science 103 ( 2017 ) 14 – 19 15
NARX networks are used both for identification 3,4,5 and control 3. Usually control engineers select the ANN architec- ture using their expert knowledge in the problem domain. In this paper we propose to use symbolic expression search method for optimal ANN structure search. Symbolic expressions search methods arose from the work 6 that introduced genetic programming method. Genetic programming is followed by cartesian genetic programming 7, grammatical evolution (GE) 8, analytic programming 9, network operator 10 etc. These methods allow to perform a search for a structure using functions as ”building blocks”. Symbolic expressions search methods are actively used in control synthesis recently 11,12 We choose grammatical evolution (GE) for optimal ANN structure 2 search. There are several works in this field 13,14. In our work we define grammatical rules of modification of some existing neural network structure. We supposed that incorporating expert knowledge into the search process is important and introduced elements of basic solution principle 11. The paper is organized as following. In section 2 control synthesis problem is formally stated. In section 3 we describe neural controller model used in the paper. Section 4 introduces the structure of the GE algorithm. In sec- tion 5 we show the neural controller performance on the simple nonlinear mobile robot kinematic model. Conclusion summarizes the paper and discusses possible directions of the future work.
2. Control system synthesis problem statement
Consider the following ODE system
x˙(t) = f (x (t) , u (t)) , (2) where x ∈ X is a system state, X ∈ Rn, u ∈ U is a control function, U ∈ Rm is closed and bounded, m < n. u(t) is defined on t0; t f . Initial conditions for (2) are given as X0 ∈ X and target terminal states set is defined as X f ∈ X, X0 ∩ X f = ∅. Generally X0 and X f are continuous, but our assumption is that if we synthesized a control ∀x(0) ∈ D ∀x( f ) ∈ D D ∈ D ∈ D D function that could move the system (2) from i X0 to j X f , X0 X0, X f X f , X0 and X f are finite and ∀x(0) ∈ given with small enough discretization step, the control function would be able to move the system (2) from i X0 ∀x( f ) ∈ to j X f : D = x(0) , x(0) ,...,x(0) , X0 0 (t0) 1 (t0) c−1(t0) (3) D = x( f ) , x( f ) ,...,x( f ) . X f 0 (t f ) 1 (t f ) d−1(t f ) (4)
Control synthesis goal is to find a function
h , x, x(0), x( f ) ∈ (t i j ) U (5)
x(0) ∈ D x( f ) ∈ D 15 that moves the system (2) from i X0 to j X f while minimizing a set of functionals in a Pareto sense p. 25 in general:
J = {J1(h), J2(h),...,Jr(h)} , (6) where = h , x, x(0), x( f ) , = , . Jk F( (t i j )) k 1 r (7) i j
h , x, x(0), x( f ) This general problem statement can always be reduced to the single functional optimization problem. As (t i j ) is nonlinear, we can approximate it with an artificial neural network. ANN architecture will be described in the next section. 16 D.E. Kazaryan and A.V. Savinkov / Procedia Computer Science 103 ( 2017 ) 14 – 19
Fig. 1. Control system with neural controller
3. Neural controller model
Artificial neural networks are great approximators. It has been proven 16 that even neural network with single hidden layer and discriminatory activation functions and suffi cient number of parameters (weights) is able to approximate any nonlinear function with required accuracy. In this paper we will use this ANN ability to approximate the desired function (5). ANN for the control system architecture shown on Figure 1 can be expressed as a function
hˆ , x, x(0), x( f ), w = f w ,... w , f w , f w , , x, x(0), x( f ) , t i j n n 3 2 2 1 1 t i j (8)
where wi, i = 0, n − 1 are subject to the parametric optimization whereas n, pi = dim wi and fi ∈ F are subjects to the structural optimization, F is the ordered set of allowed activation functions. In this paper we used fully-connected layers for ANN.
4. Grammatical evolution for neural controller synthesis
Grammatical evolution is an evolutionary algorithm that uses formal grammar rules given in a Backus — Naur form (BNF). BNF is a way for defining the language grammar in the form of production rules. Rules are formed using terminals and nonterminals. Terminals are elements of the language and nonterminals are expressions that can be replaced using production rules to other nonterminals, terminals or their combination. Using these rules GE builds possible problem solutions in a string form. The string obtained is a subject to evaluation. Usually during evaluation it should be translated or interpreted, so it is preferable to use a programming language that has built-in eval operator. For the search process GE uses a search engine, usually genetic algorithm 8 or particle swarm optimization algo- rithm 17. Search algorithm operates over a population of integer or binary arrays of variable length. During the search these arrays are transformed using operators specific to the certain algorithm. In our work genetic algorithm is used, so crossover and mutation operators are chosen. GE requires several sets to be defined: N – set of nonterminals, T – set of terminals, S – set of possible initial symbols (usually S ∈ N, but it is possible to use a predefined string, that contains at least one nonterminal), P – set of production rules. GE can be easily adopted for the structural optimization of the neural network (8). Let us define sets described above:
N = {
and P can be represented as
(1)
Fig. 2. Initial neural network structure for grammatical evolution
(2)
where
• add l(
Initial network structure for the grammar described above contains 2 dim x inputs and an output layer with dim u outputs (see Figure 2). As the program is written in Python, the string obtained is executed using eval function. One may notice that the parametric optimization is not included in the grammatical evolution process. We decided to use the genetic algorithm to perform a parametric optimization of neural networks obtained during evolution despite the fact that it significantly increased computational cost. This approach was chosen because modification-based grammatical evolution can create or destroy segments of neural networks multiple times during single array-to-string transformation, so it is not clear which weights should be used after these variations. 18 D.E. Kazaryan and A.V. Savinkov / Procedia Computer Science 103 ( 2017 ) 14 – 19
Fig. 3. Mobile robot trajectory from the initial condition (−8, −4, 0) to the final state (0, 0, 0) using control function obtained during grammatical evolution
5. Computational experiment
Consider the kinematic model of the unmanned mobile robot given in form of (2), adopted from 12:
x˙ = u1 cos θ, y˙ = u1 sin θ, (9) θ˙ = u1 , L tan u2 where x, y are coordinates of the center of vehicle mass, θ is the direction angle, L = 4 is the dimensional property of ∈ − ∈ − D = { − , − , , , − , , − , , , , , } the robot. Controls are u1 [ 5; 5], u2 [ 1; 1]. Initial conditions set is X0 ( 8 4 0) (8 4 0) ( 8 4 0) (8 4 0) . D = { , , } Terminal conditions set contains only one tuple: X f (0 0 0) . x and y are subject to state constraints that model obstacles in phase space (see Figure 3). We used the penalty function π(x, y) for state constraints violations. Quality functionals are:
= 4 (i) → , J1 i=1 t f min , ,θ ∈ D (x0i y0i 0i) X 0 (10) = 4 2 (i) + 2 (i) + θ2 (i) + π (i), (i) → , J2 i=1 x (t f ) y (t f ) (t f ) (x y ) min , ,θ ∈ D (x0i y0i 0i) X0
(i) (i) where t f is a time of reaching the goal state from the ith initial condition (if the goal is not reached, t f receives the predetermined maximal possible value), x(i) and y(i) are trajectories of the robot from the ith initial condition to the final state.
6. Conclusion
In this paper we discussed one of the most perspective branches of genetic programming and applied it to the problem of control system synthesis, where control function is approximated by the artificial neural network. We used the basic solution principle to seed grammatical evolution and developed production rules, making GE based on D.E. Kazaryan and A.V. Savinkov / Procedia Computer Science 103 ( 2017 ) 14 – 19 19 variations, although it is possible that more elementary variations would give us better results. The method produced a reproducible neural network structure along with the array of weights, tuned by genetic algorithm. Let us notice that despite this method gives good results, it required significant amount of computations. Moreover, the process of algorithm tuning is more ”trials-and-errors” than exact procedures and relies on the developer’s expe- rience. It is possible to apply any suitable optimization algorithm to adjust algorithm parameters, that would require even more computational resources. We suppose that future directions of this work should be focused on computational optimization, that will allow us to use larger sets of initial and final conditions, leading to more robust solutions.
References
1. Psaltis D, Sideris A, Yamamura AA. A multilayered neural network controller. IEEE Control Syst. Mag. 1988;8:17-21. 2. Nguyen DH, Widrow B. Neural networks for self-learning control systems. IEEE Control Syst. Mag. 1990;10:18-23. 3. Narendra KS, Parthasarathy K. Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Netw. 1990;1:4-27. 4. Trajanoski Z, Wach P. Neural predictive control for insulin delivery using the subcutaneous route. IEEE Trans. Biomed. Eng. 1998;45:1122-34. 5. Tijani IB et al. Nonlinear identification of a small scale unmanned helicopter using optimized NARX network with multiobjective differential evolution. Eng Appl Artif Intel 2014;33:99-115. 6. Koza JR. Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge: A Bradford Book; 1992. 7. Miller JF, Thomson P. Cartesian genetic programming. Proc. of European Conference on Genetic Programming 2000;1802:121-32. 8. ONeill M, Ryan C. Grammatical evolution. IEEE Trans. Evol. Comput. 2001;5:349-58. 9. Zelinka I, Oplatkova Z, Nolle L. Analytic programming – Symbolic regression by means of arbitrary evolutionary algorithms. Int J Simul Syst Sci Technol 2005;6:44-56. 10. Diveev AI, Sofronova EA. The synthesis of optimal control system by the network operator method. IFAC Proc 2009;42:232-7. 11. Diveev AI, Sofronova EA. Genetic programming method for control systems identification. Vestnik of DSTU 2010;5:623-34. 12. Diveev AI et al. Variational genetic programming for optimal control system synthesis of mobile robots. IFAC-PapersOnLine 2015;48:10611. 13. Ahmadizar F et al. Artificial neural network development by means of a novel combination of grammatical evolution and genetic algorithm. Eng Appl Artif Intel 2015;39:1-13. 14. de Campos LML, de Oliveira RCL, Roisenberg M. Optimization of neural networks through grammatical evolution and a genetic algorithm. Expert Syst Appl 2016;56:368-84. 15. Banichuk NV. Introduction to optimization of structures (Vol 1). New York: Springer New York; 1990. 16. Cybenko G. Approximation by superpositions of a sigmoidal function. Math Control Signal 1989;2:30314. 17. ONeill M and Brabazon A. Grammatical swarm: the generation of programs by social programming. Natural Computing 2006;5(4):4362.