Knowledge-Based Systems 44 (2013) 78–89 Contents lists available at SciVerse ScienceDirect Knowledge-Based Systems journal homepage: www.elsevier.com/locate/knosys Differential evolution with local information for neuro-fuzzy systems optimisation ⇑ Ming-Feng Han , Chin-Teng Lin, Jyh-Yeong Chang Institute of Electrical Control Engineering, National Chiao Tung University, 1001 University Road, Hsinchu 300, Taiwan, ROC article info abstract Article history: This paper proposes a differential evolution with local information (DELI) algorithm for Takagi–Sugeno– Received 3 October 2012 Kang-type (TSK-type) neuro-fuzzy systems (NFSs) optimisation. The DELI algorithm uses a modified Received in revised form 21 December 2012 mutation operation that considers a neighbourhood relationship for each individual to maintain the Accepted 23 January 2013 diversity of the population and to increase the search capability. This paper also proposes an adaptive Available online 4 February 2013 fuzzy c-means method for determining the number of rules and for identifying suitable initial parameters for the rules. Initially, there are no rules in the NFS model; the rules are automatically generated by the Keywords: fuzzy measure and the fuzzy c-means method. Until the firing strengths of all of the training patterns sat- Neuro-fuzzy systems (NFSs) isfy a pre-specified threshold, the process of rule generation is terminated. Subsequently, the DELI algo- Differential evolution (DE) Neuro-fuzzy systems optimisation rithm optimises all of the free parameters for NFSs design. To enhance the performance of the DELI Evolutionary algorithm (EA) algorithm, an adaptive parameter tuning based on the 1/5th rule is used for the tuning scale factor F. Optimisation The 1/5th rule dynamically adjusts the tuning scale factor in each period to enhance the search capability of the DELI algorithm. Finally, the proposed NFS with DELI model (NFS-DELI) is applied to nonlinear con- trol and prediction problems. The results of this paper demonstrate the effectiveness of the proposed NFS-DELI model. Crown Copyright Ó 2013 Published by Elsevier B.V. All rights reserved. 1. Introduction mise the sophisticated membership functions for a nonlinear water-level control system. Seng [20] proposed a neuro-fuzzy net- The popular research topic of neuro-fuzzy systems (NFSs) has work that was based on the radial basis function neural network in been successfully applied to many areas [1–10]. Such systems which all of the parameters are tuned simultaneously using a GA. bring both low-level learning and computational power of neural Juang [22] successfully applied a GA to TSK-type recurrent neu- networks into fuzzy systems and the high-level, human-like think- ro-fuzzy system design for control problems. ing and reasoning of fuzzy systems to neural networks. To train the Another category of EAs for NFSs design called particle swarm parameters for designing an NFS, many papers had employed the optimisation (PSO) appears to provide efficient and powerful backpropagation (BP) algorithm [1,8–10]. The BP algorithm is a search capability in the search space; this evolutionary computa- powerful training technique that quickly minimises the error func- tion technique was developed by Kennedy and Eberhart in 1995 tion of the NFS. However, the BP algorithm may become trapped at [42]. The underlying motivation for the development of the PSO a local optimal solution and never find the global optimal solution. algorithm is the social behaviour of animals, such as bird flocking, To overcome this disadvantage, many researchers have proposed fish schooling and swarming. The PSO algorithm has been success- NFS design that uses evolutionary algorithms (EA) [11–23]. fully applied to many optimisation problems, such as NFSs design The genetic algorithm (GA) is one of the most well-known EAs. [23–26] for control problems, and has shown improved perfor- Many researchers have developed GAs to implement fuzzy systems mance over GAs. and neuro-fuzzy systems to automate the determination of param- Hybrid EAs have been investigated in many studies [27–30]. eters and structures [14–22].The genetic fuzzy system [8–17] was Such hybrids are frequently combinations of local searches and characterised using a fuzzy system as an individual in a genetic EA s, also known memetic algorithms [29,30]. A hybrid of a cultural operator. Russo [18] applied a GA to the design of fuzzy controller method and a cooperative PSO (CPSO) was applied in the design of membership functions, in which each fuzzy rule was treated as an a functional link-based neural fuzzy network (FLNFN) [27]; this individual. Ng and Li [19] applied chromosomes in a GA to opti- method was referred to as the FLNFN–CCPSO algorithm. In the FLNFN–CCPSO algorithm, a swarm optimises only one parameter of an FLNFN. Another hybrid EA was the combination of a GA and a PSO, which is called the HGAPSO algorithm [28]. In the HGA- ⇑ Corresponding author. Tel.: +886 3 5712121. PSO algorithm, new individuals are created by a combination of a E-mail addresses: [email protected] (M.-F. Han), [email protected] (C.-T. Lin), [email protected] (J.-Y. Chang). PSO and the crossover and mutation operations of a GA. 0950-7051/$ - see front matter Crown Copyright Ó 2013 Published by Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.knosys.2013.01.023 M.-F. Han et al. / Knowledge-Based Systems 44 (2013) 78–89 79 Moreover, the differential evolution (DE) algorithm, proposed Rulej : IF x1 is A1j and x2 is A2j and ...xn is Anj 1 by Storn and Price [31,32], which is an efficient global optimiser ð Þ THEN yj ¼ w0j þ w1jx1 þ ...þ wnjxn; in the continuous search domain. The DE algorithm employs the difference of two randomly selected individuals as the source of where x1, ..., xn are the input variables, yi is the system output random variations for third individual and applies to difficult opti- variable, A1i, ..., Ani are the linguistic terms of the pre-condition misation problems [31–39,52–54]. Comparison with GA, the DE part with a Gaussian membership function, n is the number of input algorithm is faster and easier EA. Further research [40] has pro- variables, and w0j, ..., wnj are the TSK weights. posed a modified DE (MODE) algorithm for an adaptive neural fuz- The architecture of four-layered NFS is shown in Fig. 1, which is zy network (ANFN-MODE) design. This MODE algorithm provides a comprised of the input, membership function, rule, and output lay- cluster-based mutation scheme for preventing the algorithm from ers. The operation functions of the nodes in each layer of the NFS (l) becoming trapped in the local optima of the search space. In addi- are described as follows, where O denotes the output of a node tion, the MODE algorithm has been applied to locally recurrent in the lth layer: NFSs design [41]. However, the MODE algorithm lacks a self- adaptive parameter tuning strategy for obtaining more reliable 2.1.1. Layer 1—input layer convergence performance [34,39]. No computation is performed in this layer. Each node in this This paper proposes a DELI algorithm for designing a TSK-type layer corresponds to one input variable and only transmits input NFS. Initially, there are no rules in the NFS model; the fuzzy rules values directly to the next layer. That is, are automatically generated by an adaptive fuzzy c-means method. ð1Þ O ¼ xi; ð2Þ Subsequently, all free parameters are learned by the proposed DELI algorithm. In the DELI algorithm, a self-adaptive parameter tuning where i =1,2,..., n are the input variables of the NFS. strategy based on the 1/5th rule is used to adjust the tuning scale factor F effectively. In the simulation, the proposed NFS with DELI 2.1.2. Layer 2—membership function layer model (NFS-DELI) is applied to nonlinear control and prediction Each node in this layer is a membership function that corre- problems. The contributions of this paper are summarized as sponds to the linguistic label of one of the input variables in layer follows: 1. In other words, the membership value specifies the degree to which an input value belongs to a fuzzy set and is calculated in (1) An adaptive fuzzy c-means method is proposed to determine layer 2 according to the expression: the number of rules for the NFS and identify suitable initial ð2Þ 2 2 parameters for an individual in the DELI algorithm. O ¼ lij ¼ exp ðxi À mijÞ =rij ; ð3Þ (2) The proposed DELI algorithm adopts a modified mutation where j =1,2,..., M, M is the number of rules in the NFS, and operation in evaluating the global best individual, the local m and r are the centre (mean) and the width (variance) of the best individual, and randomly chosen individuals for main- ij ij Gaussian membership function of the input variable, respectively. taining the diversity of population and increasing the search capability. 2.1.3. Layer 3—Rule layer (3) We also propose a self-adaptive parameter tuning strategy This layer receives one-dimensional membership degrees of the based on the 1/5th rule to effectively adjust the tuning scale associated rule from a set of nodes in layer 2. Here, the product factor F in the DELI algorithm and balance the exploitation operator is adopted to perform the precondition part of the fuzzy ability and the exploration ability. rules. As a result, the output function of each inference node is (4) Three simulations are conducted to evaluate the perfor- mance of the NFS-DELI algorithm. Comparisons with other EAs demonstrate the superiority of the performance of the NFS-DELI model. (5) In the simulation, we use non-parametric Wilcoxon signed- rank test to verify the statistical significance.